pyexiv2-developers team mailing list archive

[Damon Lynch <damonlynch@xxxxxxxxx>]

On Fri, Jan 21, 2011 at 3:35 AM, Olivier Tilloy <olivier@xxxxxxxxxx> wrote:

>
> Out of curiosity, what’s your use case for pickling image metadata?
> Ultimately, pickling is no more than serializing data (on disk or in
> memory), and this data is already in the image itself and can be
> "reconstructed" from just the file name. Wouldn’t that work for you?
>
>
> For instance, the problem of copying photos from memory cards onto the hard
drive and renaming them. For each memory card, a process copies the photos
and reads the metadata. So if you have two memory cards, that's two
processes running in parallel. Both processes then send a message to a
daemon renaming process, whose only task is to rename photos using the exif
information, sequence numbers, and whatever else is needed. The daemon
process itself could load the metadata but it's a relatively slow operation,
and thus better to do in parallel.

My preliminary testing using multiprocessing, queues and pipes indicates
that in the case of copying and renaming photos in parallel, the scanning
phase (determining what photos are at a location and loading them into a
TreeView to show the user) takes only 6% of the time it takes to do the same
thing with threads and locks. Clearly the performance gains can be enormous.

Best,
Damon