openstack team mailing list archive

[Eric Day <eday@xxxxxxxxxxxx>]

Hi Sandy,

On Thu, Feb 24, 2011 at 07:42:34PM +0000, Sandy Walsh wrote:
> Looks like a fun project Eric. I only got caught up on the ML this weekend and I'm behind again already. 

It's a never-ending battle. I find routing all messages from jaypipes
into /dev/null helps. :)

> 1. Will broadcast queues be supported? 

As we discussed on IRC, this can be one of two this. Broadcast
(unreliable) or replication (reliable). The former is part of the
service. The latter is possible using workers from the start, but
in the future we'll probably add a special "router" API for these
specialized workers. This will look kind of like exchanges in AMQP. For
more info on both, see:

http://wiki.openstack.org/QueueService#Multi-cast_Event_Notifications
http://wiki.openstack.org/QueueService#Other_Features

> 2. What are the guarantees of idempotence in the event of retries/errors/etc? (not applicable to broadcast obviously).

Basically, for messages that were added with persistence, we error on
the side of duplicating the message instead of loosing the message. The
client can be fully aware of this. If a delete fails (server holding
the message is down), it can ignore (and duplicates are fine), revert
the work, or mark that a duplicate may be coming (mark the message
ID somewhere to check later when the duplicate may show up).

> 3. Could there be support for WAN clusters? i.e. only forward certain queues to remote locations to keep chatter down?

Sure, we can construct any topology we want. For a public queue
service, I'm thinking having a separate cluster per zone (whatever
that means for deployment), and then clients can use the local zone
or remote zone, probably preferring the local for speed.

> 4. Are all operations assumed to be one-way, or is it assumed that return values are part of each call. i.e. do I have to set up other queues for return values as callbacks or is that part of the framework? Is it optional?

Right now it's all async. A sync operation with a 60 second timeout
could be done with:

PUT /account/queue/<message_id>
GET /account/queue/<message_id>_result?wait=60

And you would write your worker to PUT the result in a message named
<message_id>_result.

-Eric