phatch-dev team mailing list archive

[Stani <spe.stani.be@xxxxxxxxx>]

Hi All,

I was happily surprised to see the multiprocessing branch of Ido. I
like to quote him from the blueprint so we can further discuss it
here:
https://blueprints.launchpad.net/phatch/+spec/multiprocessing-support
"I took a stab at prototyping/hacking MP into Phatch and I got a very
crude version working by just modifying api.py:apply_actions. With a
simple action list of 'round' + 'scale' + 'save' on 135 images, it
took 0:02:43 seconds on a dual-core machine with 4 worker threads
compared to 0:04:25 seconds on a non-modified version of Phatch.
Considering how 'easy' hacking it was I would think that making a
cleaner patch would not take too much time. Since I got away with
basically only changing how apply_actions works it would same that
multi-processing capabilities could be a settings/switch potentially
disabled by default until it would be proven stable. (I have not
tested this with anything else then an Ubuntu/Debian Linux system) I
cannot see a place here to attach some of the files but I pushed the
hack into a branch for others to look at. (The code/branch is not
intended to be pushed to trunk, it's just my little attempt to see how
much time a very basic implementation would take.)"

The code is here:
http://bazaar.launchpad.net/~v-oostveen/phatch/multiprocessing/annotate/head:/phatch/core/api.py

What I like is that the intervention is quite minimal. So it looks
like we will have multiprocessing in Phatch soon. What in my opinion
nows to be done is:
1) refactor the code so multiprocessing is optional, so it can live
side by side without multiprocessing
2) optimize step by step (profiling, sharing data if needed)

If step 1 is properly implemented and stable, I see no reason to wait
to merge with trunk. The sooner the better. It is too early to say if
it will make it for Phatch 0.2 but who knows. The living side by side
is necessary as Phatch still has to support python 2.4 Or if a good
backport is available we might consider it as well. In that case I'd
prefer to ship this backport with Phatch in phatch/other.
http://pyprocessing.berlios.de/
http://code.google.com/p/python-multiprocessing/

Ido and Erich, what is your opinion on that?

Ido, thank you very much for this contribution. Many people suggested
multiprocessing, but unfortunately I don't have the time for it. As it
is so obvious what the benefits for Phatch would be, I hoped that
someone else would step forward. So I am really glad you did it.
Erich, feel free to join and give Ido a hand. If you don't have much
time, only your feedback would be welcome as well.

I am really thrilled to see so many people actively contributing to
Phatch. Phatch needs you!

Robin, can you test this branch on the Mac (needs python2.6)?
https://code.launchpad.net/~v-oostveen/phatch/multiprocessing

Have a nice weekend,
Stani
-- 
Phatch Photo Batch Processor - http://photobatch.stani.be
SPE Python IDE - http://pythonide.stani.be