launchpad-dev team mailing list archive
-
launchpad-dev team
-
Mailing list archive
-
Message #06098
Re: Design pattern for interruptible processing of continuous data
One thing (of many :)) that I don't understand here.
You are saying that figuring out what packages need new diffs made
will be very expensive (so you can't just run in a check-and-do mode.
But you're also saying that storing a simple index into the
work-to-be-done is all the abstraction you need? That seems to imply
that generating a list of packages that need diffs made after your
timestamp/db-id is cheap to do.
These two things seem to be in contradiction to each other.
Separately, as far as modelling stuff now to fit e.g. rabbit later,
model a queue which you put work into and dispatch work from : its
what rabbit does, and what the existing TaskSystem does. (I'd probably
use the TaskSystem for anything new-and-urgent).
-Rob
Follow ups
References