← Back to team overview

launchpad-dev team mailing list archive

Re: Design pattern for interruptible processing of continuous data

 

On Thursday 06 January 2011 04:56:24 Robert Collins wrote:
> One thing (of many :)) that I don't understand here.
> 
> You are saying that figuring out what packages need new diffs made
> will be very expensive (so you can't just run in a check-and-do mode.
> 
> But you're also saying that storing a simple index into the
> work-to-be-done is all the abstraction you need? That seems to imply
> that generating a list of packages that need diffs made after your
> timestamp/db-id is cheap to do.
> 
> These two things seem to be in contradiction to each other.

I've no idea how you can come to that conclusion - one is an item of work and 
one is a point of reference as to which piece of work to start next.

There's absolutely no contradiction there.

> Separately, as far as modelling stuff now to fit e.g. rabbit later,
> model a queue which you put work into and dispatch work from : its
> what rabbit does, and what the existing TaskSystem does. (I'd probably
> use the TaskSystem for anything new-and-urgent).

I probably need to speak to you in realtime, I think you're completely 
misunderstanding what needs to happen here.

J.



Follow ups

References