maas-devel team mailing list archive

[Gavin Panella <gavin.panella@xxxxxxxxxxxxx>]

On 4 November 2013 07:25, Julian Edwards <julian.edwards@xxxxxxxxxxxxx> wrote:
> It came out in June last year but seems not to be packaged as the celery
> in Saucy is still on the same version that was distributed with Quantal.
>
> There's a few nice changes:
>
> - new api (old one still available)
> - chaining of tasks now possible, also with callbacks on groups of tasks
> - uses librabbitmq C lib instead of py-amqp (one fewer Python dependency)
> - threadless; performs better
> - Now uses a single "celery" command line which wraps up some
> interesting management commands.
> - runs on Python 3
>
> Also while I am talking about celery, I noticed that it has fanout
> queues called "Broadcast" queues.  This will enable us to easily solve
> the region worker SPoF problem.
>
> Sadly there doesn't seem to be anything built-in to detect missing
> workers, so we'll probably have to DIY that.

Given that RabbitMQ and Celery pose problems to HA, it may be worth
considering their replacement with ZeroMQ. The HA problems are not
insurmountable, but they do represent work that needs doing, and
debugging, and maintenance, and so forth. Every service that we can
get rid of from MAAS is one less thing for end users to administer and
maintain too.

RabbitMQ is also a large moving part that we don't understand very
well. Moving to something like ZeroMQ would, I hope, reduce the
cognitive load on developers and users alike. Celery 3.1 (in
development) has hints of ZeroMQ compatibility as a transport so this
might be something that we can do in stages, or just stop after
eliminating RabbitMQ from the stack.

I'll collect the ways in which we're currently using RabbitMQ and
Celery and come up with ways to replace them with ZeroMQ with Celery
3.1+ (or another task runner, or something home-grown), and estimate
the work needed.

Gavin.