launchpad-dev team mailing list archive
-
launchpad-dev team
-
Mailing list archive
-
Message #06101
Re: Design pattern for interruptible processing of continuous data
On Thursday 06 January 2011 04:56:24 Robert Collins wrote:
> One thing (of many :)) that I don't understand here.
>
> You are saying that figuring out what packages need new diffs made
> will be very expensive (so you can't just run in a check-and-do mode.
>
> But you're also saying that storing a simple index into the
> work-to-be-done is all the abstraction you need? That seems to imply
> that generating a list of packages that need diffs made after your
> timestamp/db-id is cheap to do.
>
> These two things seem to be in contradiction to each other.
I've no idea how you can come to that conclusion - one is an item of work and
one is a point of reference as to which piece of work to start next.
There's absolutely no contradiction there.
> Separately, as far as modelling stuff now to fit e.g. rabbit later,
> model a queue which you put work into and dispatch work from : its
> what rabbit does, and what the existing TaskSystem does. (I'd probably
> use the TaskSystem for anything new-and-urgent).
I probably need to speak to you in realtime, I think you're completely
misunderstanding what needs to happen here.
J.
Follow ups
References