openstack team mailing list archive

[Doug Hellmann <doug.hellmann@xxxxxxxxxxxxx>]

On Fri, May 18, 2012 at 4:42 AM, Nick Barcet <nick.barcet@xxxxxxxxxxxxx>wrote:

> Hello everyone,
>
> Next week's irc meeting will have for goal to choose a reference
> messaging queue service for the ceilometer project.  For this meeting to
> be able to be successful, a discussion on the choices that we have to
> make need to occur first right here.
>
> To open the discussion here are a few requirements that I would consider
> important for the queue to support:
>
> a) the queue must guaranty the delivery of messages.
> To the contrary of monitoring, loss of events may have important billing
> impacts, it therefore cannot be an option that message be lost.
>
> b) the client should be able to store and forward.
> As the load of system or traffic increases, or if the client is
> temporarily disconnected, client element of the queue should be able to
> hold messages in a local queue to be emitted as soon as condition permit.
>
> c) client must authenticate
> Only client which hold a shared private key should be able to send
> messages on the queue.
>

Does the username/password authentication of rabbitmq meet this requirement?


>
> d) queue may support client signing of individual messages
> Each message should be individually signed by the agent that emits it in
> order to guaranty non repudiability.  This function can be done by the
> queue client or by the agent prior to en-queuing of messages.
>

We can embed the message signature in the message, so this requirement
shouldn't have any bearing on the bus itself. Unless I'm missing something?


>
> d) queue must be highly available
> the queue servers must be able to support multiple instances running in
> parallel in order to support continuation of operations with the loss of
> one server.  This should be achievable without the need to use complex
> fail over systems and shared storage.
>
> e) queue should be horizontally scalable
> The scalability of queue servers should be achievable by increasing the
> number of servers.
>
> Not sure this list is exhaustive or viable, feel free to comment on it,
> but the real question is: which queue should we be using here?
>

While I see the benefit of discussing requirements for the message bus
platform in general, I'm not sure we need to dictate a specific
implementation. If we say we are going to use the nova RPC library to
communicate with the bus for sending and receiving messages, then we can
use all of the tools for which there are drivers in nova -- rabbit, qpid,
zeromq (assuming someone releases a driver, which I think is being worked
on somewhere), etc. This leaves the decision of which bus to use up to the
deployer, where the decision belongs. It also means we won't end up
choosing a tool for which the other projects have no driver, leading us to
have to create one and add a new dependency to the project.

Doug