openstack team mailing list archive

[Eric Day <eday@xxxxxxxxxxxx>]

Hi Mike,

On Sat, Feb 19, 2011 at 07:42:12PM -0600, Michael Barton wrote:
> On Fri, Feb 18, 2011 at 9:38 PM, Eric Day <eday@xxxxxxxxxxxx> wrote:
> > Hi Mike,
> >
> > You make a good point, I apologize for not documenting some of the
> > ideas sooner. The architecture I had in mind borrows from other
> > queue systems I've worker on, along with feedback I've gotten from
> > various conversations while getting things started. For the hashing,
> > we could possibly reuse the distribution ring algorithm swift uses for
> > selecting server sets from accounts. I've added some sample deployment
> > descriptions to the wiki page:
> >
> > http://wiki.openstack.org/QueueService#Deployment_Configurations
> 
> 
> That's cool, thanks for humoring me on that a bit.  I was worried that
> I hadn't seen anything suggesting this queue could meet some of our
> expected needs.  FWIW, I think the basic architecture is probably
> good.  It'll be interesting to see how different queue features can
> interact with CAP theorem.

The goal is AP, but of course do our best with C when possible to
keep the UX acceptable. For this service it comes down to workers
being idempotent, which we'll need to state clearly for users. Of
course, all work being pushed through the queue doesn't need to be
idempotent, there are various application-specific ways workers can
deal with duplicates when needed.

-Eric