openstack team mailing list archive

[Craig Vyvial <cp16net@xxxxxxxxx>]

You can allow nova to send notifications to multiple topics by setting this
option in your nova.conf.

###### (ListOpt) AMQP topic used for Nova notifications
notification_topics="notifications,metering,monitoring"

This will allow you to consume all the messages from a different service.

On Wed, May 9, 2012 at 10:34 AM, Doug Hellmann
<doug.hellmann@xxxxxxxxxxxxx>wrote:

>
>
> On Wed, May 9, 2012 at 6:42 AM, Kiall Mac Innes <kiall@xxxxxxxxxxxx>wrote:
>
>> Kinda! The queue has a name, but that name has no bearing on the set of
>> messages received.
>
>
>> If you create a queue called "MyCustomNotificationQueue", you can bind
>> that to the "notifications" exchange using the "notifications.info"
>> routing key.
>>
>> (I'm guessing some of the names here.. I know AMQP, and not the specific
>> naming nova uses!)
>>
>
> notifications.info is right.
>
>
>>
>> Nova just happens to use the same queue name and routing key. I believe
>> this is causing the confusion.
>>
>
> Exactly. I ended up creating a separate queue for each client that I have
> and setting them to auto-delete when the client disconnects. That way I can
> have as many clients connecting and listening as I want. The code is in
> https://github.com/dhellmann/metering-prototype if you want to take a
> look.
>
>
>>
>
>>
>> Does this make sense?
>>
>> Anyway - The RabbitMQ docs probably explain it better than I..
>> http://www.rabbitmq.com/tutorials/tutorial-five-python.html
>>
>> Thanks,
>> Kiall
>>
>>
>>
>> On Wed, May 9, 2012 at 11:30 AM, Day, Phil <philip.day@xxxxxx> wrote:
>>
>>> OK, get that so far – so both consumers need to declare and use the same
>>> exchange.****
>>>
>>> ** **
>>>
>>> But If I understand the next step right, to get multiple consumers of
>>>  info notification messages they would all need to create separate “
>>> notifications.info” queues into that exchange.    And isn’t that
>>> exactly what Nova currently does to create a shared queue ?****
>>>
>>> ** **
>>>
>>> Phil****
>>>
>>> ** **
>>>
>>> *From:* Kiall Mac Innes [mailto:kiall@xxxxxxxxxxxx]
>>> *Sent:* 09 May 2012 10:51
>>> *To:* Day, Phil
>>> *Cc:* openstack@xxxxxxxxxxxxxxxxxxx; Russell Bryant; Doug Hellmann
>>>
>>> *Subject:* Re: [Openstack] [nova] why does notification use a "topic"
>>> exchange instead of "fanout"?****
>>>
>>> ** **
>>>
>>> Your own queue listener should attempt to declare the exchange, using
>>> the same settings as Nova does. ****
>>>
>>> If the exchange exists, its a noop. Otherwise it's created for you.****
>>>
>>> After that, if you start up Nova, it will do the same and reuse your
>>> exchange.****
>>>
>>> Obviously this works both ways, and either nova or your code can declare
>>> the exchange.****
>>>
>>> AMQP is designed to be a configuration-less protocol, where resources
>>> are configured by the first consumer attempting to use them.****
>>>
>>> Thanks,
>>> Kiall****
>>>
>>> Sent from my phone.****
>>>
>>> On May 9, 2012 9:52 a.m., "Day, Phil" <philip.day@xxxxxx> wrote:****
>>>
>>> Hi Doug,****
>>>
>>>  ****
>>>
>>> > I think you missed my main point, which was that a topic exchange does
>>> > not impose a limitation that only one client can consume a given
>>> > notification.  That's only true if each client is consuming from the
>>> > same queue bound to the exchange.****
>>>
>>>  ****
>>>
>>> So just to be clear, if I understand you correctly within the nova
>>> service/rpc abstraction layers the code is set up so that all services do
>>> bind to the same queue, and hence we get the round-robin delivery.****
>>>
>>> But, if someone wanted to write a separate notification consumer so that
>>> they didn’t block anyone else from seeing the same messages then they (the
>>> consumer) should create a new queue on the existing topic exchange.****
>>>
>>>  ****
>>>
>>> Is that correct – and is there any worked example of doing this ?****
>>>
>>>  ****
>>>
>>> I thought within the nova code both the exchange and topic queues were
>>> set up by the consumer (so for example all compute_managers try to create
>>> the “compute” exchange and topic queue, but its only created by the first
>>> one and the others connect to the same queue).   In that context I’m
>>> finding it hard to see how to change this model to have multiple “
>>> notify.info” topic queues into the same exchange ?****
>>>
>>>  ****
>>>
>>> Cheers,****
>>>
>>> Phil****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>> *From:* openstack-bounces+philip.day=hp.com@xxxxxxxxxxxxxxxxxxx [mailto:
>>> openstack-bounces+philip.day=hp.com@xxxxxxxxxxxxxxxxxxx] *On Behalf Of *Doug
>>> Hellmann
>>> *Sent:* 08 May 2012 23:34
>>> *To:* Russell Bryant
>>> *Cc:* openstack@xxxxxxxxxxxxxxxxxxx
>>> *Subject:* Re: [Openstack] [nova] why does notification use a "topic"
>>> exchange instead of "fanout"?****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>> On Tue, May 8, 2012 at 6:04 PM, Russell Bryant <rbryant@xxxxxxxxxx>
>>> wrote:****
>>>
>>> On 05/08/2012 05:59 PM, Doug Hellmann wrote:
>>> >     Here is a relevant section pulled out of the amqp 0-9-1 spec:
>>> >
>>> >        3.1.3.3 The Topic Exchange Type
>>> >
>>> >        The topic exchange type works as follows:
>>> >
>>> >            1. A message queue binds to the exchange using a routing
>>> >               pattern, P.
>>> >            2. A publisher sends the exchange a message with the routing
>>> >               key R.
>>> >            3. The message is passed to the message queue if R matches
>>> P.
>>> >
>>> >        The routing key used for a topic exchange MUST consist of zero
>>> or
>>> >        more words delimited by dots. Each word may contain the letters
>>> A-Z
>>> >        and a-z and digits 0-9.
>>> >
>>> >        The routing pattern follows the same rules as the routing key
>>> with
>>> >        the addition that * matches a single word, and # matches zero or
>>> >        more words. Thus the routing pattern *.stock.# matches the
>>> routing
>>> >        keys usd.stock and eur.stock.db but not stock.nasdaq.
>>> >
>>> >     In nova, for a given topic such as 'scheduler', all of the
>>> consumers are
>>> >     binding to the same queue on the topic exchange, resulting in
>>> >     round-robin delivery to each of the consumers.  If instead you
>>> make a
>>> >     new queue, you can get your own copy of each message.
>>> >
>>> >     There is an additional benefit of using a topic exchange here.  The
>>> >     topic used for notifications is 'notifications.<priority>'.  That
>>> means
>>> >     that when you create your queue, you can set it up to receive all
>>> >     notifications, or only notifications of a certain priority.
>>> >
>>> >
>>> > Topic exchanges make a lot of sense for messages that should only be
>>> > consumed once, such as tasks. Notifications are different. Lots of
>>> > different clients might want to know that some event happened in the
>>> > system. The way things are in Nova today, they can't. The first client
>>> > who consumes a notification message will prevent all of the other
>>> > clients from seeing that message at all.****
>>>
>>> I think you missed my main point, which was that a topic exchange does
>>> not impose a limitation that only one client can consume a given
>>> notification.  That's only true if each client is consuming from the
>>> same queue bound to the exchange.****
>>>
>>>  ****
>>>
>>> Yes, that wasn't obvious from any of the kombu documentation I've seen
>>> so far. I'll keep looking.****
>>>
>>>  ****
>>>
>>> Thanks,****
>>>
>>> Doug****
>>>
>>>  ****
>>>
>>>
>>> > I can change Nova's notification system to use a fanout exchange (in
>>> > impl_kombu.py changing the exchange type used by NotifyPublisher), but
>>> > before I submit a patch I want to make sure the current implementation
>>> > using a topic exchange wasn't selected deliberately for some reason.**
>>> **
>>>
>>> I think using a fanout exchange would be a downgrade.  As I mentioned
>>> before, a topic exchange allows you to create a queue to get all
>>> notifications or only notifications of a specific priority.  If the
>>> exchange type is changed to fanout, it's everybody gets everything, and
>>> that's it.
>>>
>>> --
>>> Russell Bryant****
>>>
>>>  ****
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp****
>>>
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>