openstack team mailing list archive

[Doug Hellmann <doug.hellmann@xxxxxxxxxxxxx>]

On Tue, May 22, 2012 at 4:05 AM, Endre Karlson <endre.karlson@xxxxxxxxx>wrote:

> If I'm understanding this correctly, the Collector is kind of like a Agent
> in Qantum (It sits on a machine doing stuff and passing info upstream).
>
> If you look at the approach they have now in Quantum Agent it's writing
> directly to the DB. But looking at the next version they seem to be moving
> to having the Agent send data upstream to the Plugin in Quantum. Why not do
> something similar?
>
> I mean if you have a MQ cluster in a deployment I think it makes more
> sense to have 1 thing that handles the db stuff then having each Collector
> connect to the db..
>

That was the goal, but I may have swapped the terminology around. For
ceilometer, the "agent" runs on the compute node and writes only to the
message queue. The "collector" runs in a central location and writes to the
database. The number of collectors you need will depend on the number of
messages being generated, but the architecture supports running several in
parallel in a way that each instance does not need to be aware of the
others.

Doug


> Endre.
> 2012/5/22 Nick Barcet <nick.barcet@xxxxxxxxxxxxx>
>
>> On 05/21/2012 10:52 PM, Doug Hellmann wrote:
>> > I have written up some of my thoughts on a proposed design for
>> > ceilometer in the wiki [1]. I'm sure there are missing details, but I
>> > wanted to start getting ideas into writing so they could be discussed
>> > here on the list, since I've talked about different parts with a couple
>> > of you separately.
>> >
>> > Let me know what you think, and especially if I am not clear or have
>> > left out any details.
>> >
>> > Thanks,
>> > Doug
>> >
>> > [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1
>>
>> Thanks a lot for putting this together Doug.
>>
>> A few questions:
>>
>> * "The collector runs on one or more central management servers to
>> monitor the message queues (for notifications and for metering data
>> coming from the agent). Notification messages are processed and turned
>> into metering messages and sent back out onto the message bus using the
>> appropriate topic. Metering messages are written to the data store
>> without modification."
>> -> Is the reason behind why collectors do not write directly to the
>> database a way to allow db less implementations as Francis suggested
>> earlier?  In this case it may be useful to say it explicitly.
>>
>> * "Plugins may require configuration options, so when the plugin is
>> loaded it is asked to add options to the global flags object, and the
>> results are made available to the plugin before it is asked to do any
>> work."
>> -> I am not sure where the "global flags object" resides and how option
>> are populated.  I think it would make sense for this to be globally
>> controlled, and therefore may require for a simple discovery exchange on
>> the queue to retrieve values and set defaults if it does not exist yet.
>>
>> * "Metering messages are signed using the hmac module in Python's
>> standard library. A shared secret value can be provided in the
>> ceilometer configuration settings. The messages are signed by feeding
>> the message key names and values into the signature generator in sorted
>> order. Non-string values are converted to unicode and then encoded as
>> UTF-8. The message signature is included in the message for verification
>> by the collector."
>> -> The signature is also kept in the database for future audit
>> processes, maybe worth mentioning it here.
>> -> In addition to a signature, I think we would need a sequence number
>> to be embedded by the agent for each message sent, so that loss of
>> messages, or forgery of messages, can be detected by the collector and
>> further audit process.
>>
>> Thanks again,
>> Nick
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>