openstack team mailing list archive

[Nick Barcet <nick.barcet@xxxxxxxxxxxxx>]

On 06/13/2012 01:55 PM, Doug Hellmann wrote:
> 
> 
> On Wed, Jun 13, 2012 at 6:56 AM, Nick Barcet <nick.barcet@xxxxxxxxxxxxx
> <mailto:nick.barcet@xxxxxxxxxxxxx>> wrote:
> 
>     On 06/12/2012 05:51 PM, Doug Hellmann wrote:
>     > See https://review.stackforge.org/163 for the code.
>     >
>     > On Tue, Jun 12, 2012 at 11:05 AM, Doug Hellmann
>     > <doug.hellmann@xxxxxxxxxxxxx <mailto:doug.hellmann@xxxxxxxxxxxxx>
>     <mailto:doug.hellmann@xxxxxxxxxxxxx
>     <mailto:doug.hellmann@xxxxxxxxxxxxx>>> wrote:
>     >
>     >     I am working on bug 1006120, adding metadata to the
>     instance-related
>     >     metering events. It's easy to add location data like availability
>     >     zone to the pollsters because the agent has an object from the
>     >     database with all of the information about the instance (we will
>     >     have to change that, but presumably we will be able to get the
>     data
>     >     via RPC, too). The notification plugins however, only have the
>     data
>     >     in the incoming message, and those messages don't include all
>     of the
>     >     data about the instance. My current plan is to have the collector
>     >     code that processes notifications look up the instance in the
>     >     database to get the rest of the data.
>     >
>     >     Does anyone have any thoughts on this plan?
>     >
>     >     https://bugs.launchpad.net/ceilometer/+bug/1006120
> 
>     Well, this seems a bit inelegant to me to have collectors to additional
>     lookup in an external DB where otherwise it's role would be limited to
>     handle and store event.  It would definitely add quite a bit of strain
>     to the code on the central collector and what would happen if the nova
>     DB is not available or overloaded?
> 
> 
> I want to eventually move off of direct DB access and ask for details
> like that via RPC, so this is just a temporary situation.
>  
> 
> 
>     I'd rather have all collection of data done from the agents rather than
>     introduce this lookup on the central collector node.  What would prevent
>     us from doing it this way?
> 
> 
> That moves the problem even further away from the database. The agent
> runs on the compute node, and we *know* we won't have DB access there by
> the time we're done with Folsom (even nova is eliminating direct DB
> access by the compute node). The agent does not process notification
> events (it only polls) because of the round-tripping of messages through
> the message bus. Even if we move the notification handling into the
> agent, it doesn't have the necessary details so we would still need to
> ask for them somehow.

Yes, I see your point.  Bad idea.

> There are a couple of other alternatives:
> 
> 1. We could move notification handling into its own daemon to get it out
> of the collector. This new daemon would still run on a central service,
> and would need to be set up to support load balancing just as the
> collector is now. The similarities are why we left the two types of
> processing in the same process to begin with.

RPC seems like a better solution for this, doesn't it?

> 2. We could modify nova to include more details about an instance when
> it publishes a notification. This is the best solution from our
> perspective, and I would like to see all of the properties anyway, but I
> don't know if there is a performance-related reason for leaving out some
> details.

That would be definitely best.  Any nova dev willing to comment here?

Nick

Attachment: signature.asc
Description: OpenPGP digital signature