← Back to team overview

nova-orchestration team mailing list archive

Re: zookeeper and eventlet

 

That was based on my verbal feedback during the summit.

I do quite a bit of work with Celery, and have been closely tracking it's development from 2.2 to 2.4 - watching it start to develop some of the features that could lead to workflow management. Ask Solem does a good job with the core functionality of celery, but I've also found frequent bugs and some "WTF" inconsistencies in how it handles background tasks. I'm fairly certain that there is no CI or integrated testing for Celery, or maybe that the tests are run irregularly, leading to periodic point releases that break functionality. 

The real killer for our use/intention is that errors deep in the structure - exceptions thrown, etc - don't propogate up well or consistently, leaving the unsure of what's happened and how to resolve it.  I've found Celery to be very effective at single tasks, but the use of subtasks, chord, and task sets (http://celery.readthedocs.org/en/latest/userguide/tasksets.html) is still pretty raw in actual functionality. I've had far more success wrapping my own (simple) orchestration logic around celery background tasks to use that as a simple "background/asynchronous task" mechanism.

I'm really not trying to dis on Celery (Ask's work) - He has been very helpful in answering questions and responding to bugs. I would assert that other options built for workflow up front behave far more consistently. 

-joe

On Nov 21, 2011, at 10:03 AM, Yun Mao wrote:
> Gary,
> 
> I think during one of the early meetings, Sandy mentioned that celery
> was not a great fit for the problem. I don't remember why though. :)
> 
> Thanks,
> Yun
> 
> On Mon, Nov 21, 2011 at 3:18 AM, Gary Kotton <GaryK@xxxxxxxxxxx> wrote:
>> Hi,
>> Is anyone familiar with Celery? http://celeryproject.org/. The fact that
>> it works with the RabbitMQ could be an advantage.
>> Thanks
>> Gary
>> 
>> -----Original Message-----
>> From: nova-orchestration-bounces+garyk=radware.com@xxxxxxxxxxxxxxxxxxx
>> [mailto:nova-orchestration-bounces+garyk=radware.com@xxxxxxxxxxxxxxxxxxx
>> ] On Behalf Of Yun Mao
>> Sent: Friday, November 11, 2011 9:44 PM
>> To: nova-orchestration@xxxxxxxxxxxxxxxxxxx
>> Subject: [Nova-orchestration] zookeeper and eventlet
>> 
>> Hi folks,
>> 
>> I've got zookeeper and eventlet working together nicely. The code is
>> available at
>> 
>> https://github.com/maoy/python-evzookeeper
>> 
>> Thanks,
>> 
>> Yun
>> 
>> --
>> Mailing list: https://launchpad.net/~nova-orchestration
>> Post to     : nova-orchestration@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~nova-orchestration
>> More help   : https://help.launchpad.net/ListHelp
>> 
> 
> -- 
> Mailing list: https://launchpad.net/~nova-orchestration
> Post to     : nova-orchestration@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nova-orchestration
> More help   : https://help.launchpad.net/ListHelp



Follow ups

References