← Back to team overview

nova-orchestration team mailing list archive

Re: zookeeper and eventlet

 

Thanks for the input.

-----Original Message-----
From: Joseph Heck [mailto:heckj@xxxxxx] 
Sent: Tuesday, November 22, 2011 5:59 PM
To: Yun Mao
Cc: Gary Kotton; nova-orchestration@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Nova-orchestration] zookeeper and eventlet

That was based on my verbal feedback during the summit.

I do quite a bit of work with Celery, and have been closely tracking
it's development from 2.2 to 2.4 - watching it start to develop some of
the features that could lead to workflow management. Ask Solem does a
good job with the core functionality of celery, but I've also found
frequent bugs and some "WTF" inconsistencies in how it handles
background tasks. I'm fairly certain that there is no CI or integrated
testing for Celery, or maybe that the tests are run irregularly, leading
to periodic point releases that break functionality. 

The real killer for our use/intention is that errors deep in the
structure - exceptions thrown, etc - don't propogate up well or
consistently, leaving the unsure of what's happened and how to resolve
it.  I've found Celery to be very effective at single tasks, but the use
of subtasks, chord, and task sets
(http://celery.readthedocs.org/en/latest/userguide/tasksets.html) is
still pretty raw in actual functionality. I've had far more success
wrapping my own (simple) orchestration logic around celery background
tasks to use that as a simple "background/asynchronous task" mechanism.

I'm really not trying to dis on Celery (Ask's work) - He has been very
helpful in answering questions and responding to bugs. I would assert
that other options built for workflow up front behave far more
consistently. 

-joe

On Nov 21, 2011, at 10:03 AM, Yun Mao wrote:
> Gary,
> 
> I think during one of the early meetings, Sandy mentioned that celery
> was not a great fit for the problem. I don't remember why though. :)
> 
> Thanks,
> Yun
> 
> On Mon, Nov 21, 2011 at 3:18 AM, Gary Kotton <GaryK@xxxxxxxxxxx>
wrote:
>> Hi,
>> Is anyone familiar with Celery? http://celeryproject.org/. The fact
that
>> it works with the RabbitMQ could be an advantage.
>> Thanks
>> Gary
>> 
>> -----Original Message-----
>> From:
nova-orchestration-bounces+garyk=radware.com@xxxxxxxxxxxxxxxxxxx
>>
[mailto:nova-orchestration-bounces+garyk=radware.com@xxxxxxxxxxxxxxxxxxx
>> ] On Behalf Of Yun Mao
>> Sent: Friday, November 11, 2011 9:44 PM
>> To: nova-orchestration@xxxxxxxxxxxxxxxxxxx
>> Subject: [Nova-orchestration] zookeeper and eventlet
>> 
>> Hi folks,
>> 
>> I've got zookeeper and eventlet working together nicely. The code is
>> available at
>> 
>> https://github.com/maoy/python-evzookeeper
>> 
>> Thanks,
>> 
>> Yun
>> 
>> --
>> Mailing list: https://launchpad.net/~nova-orchestration
>> Post to     : nova-orchestration@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~nova-orchestration
>> More help   : https://help.launchpad.net/ListHelp
>> 
> 
> -- 
> Mailing list: https://launchpad.net/~nova-orchestration
> Post to     : nova-orchestration@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nova-orchestration
> More help   : https://help.launchpad.net/ListHelp



References