← Back to team overview

pyexiv2-developers team mailing list archive

Re: Pickling and multiprocessing

 

Hi Damon,

On 2011-01-21, Damon Lynch <damonlynch@xxxxxxxxx> wrote:
> Hi all,
> 
> I got this error when I tried to pickle a class that included the
> pyexiv2 metadata:
> 
> RuntimeError: Pickling of "libexiv2python._Image" instances is not
> enabled (http://www.boost.org/libs/python/doc/v2/pickle.html)
> 
> Pickling is essential when passing data from one process to another in
> python multiprocessing. The page linked above lists a few pitfalls,
> however. I don't know if they are relevant to the project or not.
> 
> Olivier do you have any plans to implement pickling support?

Pickling support was implemented for tags in 0.3 (see http://pad.lv/628735).
However pickling an ImageMetadata itself is not implemented indeed. Its
implementation is not planned, but feel free to submit a bug report to
track the issue.

Out of curiosity, what’s your use case for pickling image metadata?
Ultimately, pickling is no more than serializing data (on disk or in
memory), and this data is already in the image itself and can be
"reconstructed" from just the file name. Wouldn’t that work for you?

> Another question: is anyone aware of any reason why reading metadata
> from different files via pyexiv2.metadata.ImageMetadata in multiple
> processes simultaneously could case a problem? I don't know that there
> is a problem, but it occurred to me that it might be important to ask in
> case there is.

I’m not aware of such a problem, if it exists. As long as you’re not
trying to read the metadata off the same file at the same moment from
two different processes, you should be safe.
Note that since 0.2, pyexiv2 correctly releases the python GIL while
reading the image (see http://pad.lv/372321), so that shouldn’t block
your processes.

> 
> Best,
> Damon

Cheers,

Olivier



Follow ups

References