← Back to team overview

launchpad-dev team mailing list archive

Re: generic job queue cronjob?

 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/21/2010 04:25 AM, Julian Edwards wrote:
> On Friday 17 September 2010 14:51:33 Aaron Bentley wrote:
>>> Cron jobs have served us well so far, but isn't it about time we stopped
>>> using them and get RabbitMQ sorted out instead?
>>
>> It's not clear what the best approach would be.  It sounds like you're
>> suggesting an approach that would require us to write and maintain our
>> own daemon, but there are solutions out there that provide their own
>> daemons.
> 
> I'm not suggesting anything (yet).

Okay. From what you said, it sounded like you wanted to use RabbitMQ,
and not something else, such as Gearman's built-in messaging, for
dispatching tasks.

>> That's why I've been gathering requirements at
>> https://dev.launchpad.net/Foundations/NewTaskSystem/Requirements to
>> ensure we don't pursue a dead end.
> 
> I think this is great; however it doesn't diminish the separate need for a way 
> of passing messages between components.

Selecting a way of passing messages is a critical part of satisfying the
New Task System requirements. Celery, Gearman and raw RabbitMQ are
separate messaging approaches.  So out of this process, you will get a
messaging solution, but I'm hesitant to use RabbitMQ for Jobs before we
have selected our solution, because it may turn out to be waste.

I have been assuming that dispatching tasks (e.g. Jobs, Builds) was the
primary use case for RabbitMQ. You say we have a separate need for
passing messages between components.  Can you give me examples of ways
you'd like to use RabbitMQ that aren't related to tasks?

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkyYqkgACgkQ0F+nu1YWqI2pdACdGz9DALqA5FHEttvZndEXpSsK
iWEAoIXvj0Hr+GFv3Tuv1uPAayES6JO/
=nwjU
-----END PGP SIGNATURE-----



Follow ups

References