launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Wed, Jan 5, 2011 at 7:24 AM, Clint Byrum <clint.byrum@xxxxxxxxxxxxx> wrote:
> I'm curious why one can't just start using message queues on the batch
> job only.
>
> Rather than a cron job that does all the work, the batch job could
> simply push all the work into a queue. Whenever the message queue is
> ready for frontend consumption, the batch jobs go away and the frontend
> starts feeding the backend directly.
>
> Trying to emulate the queue's robustness seems a noble, but possibly
> unnecessary effort if queues are coming any time soon.

Me too, and also I'd like to know about the actual use cases being
designed for; the idea of a single fixed sequence marker across the
system worries me from a performance and scaling aspect: it creates a
bottleneck in the system that we should demand a large win for having.
(Compare other bottleneck things like the code import machine table
which have regularly experienced contention issues).

-Rob