openstack team mailing list archive

[Doug Hellmann <doug.hellmann@xxxxxxxxxxxxx>]

On Tue, May 22, 2012 at 12:53 PM, Matt Joyce <matt.joyce@xxxxxxxxxxxxxxxx>wrote:

> My point of concern.
> \
> If an agent is being built into the compute nodes, that would best be a
> split out project.
>
> Two major reasons.  First and foremost sub projects should not be spinning
> up their own agents.  Secondly, there is a use case of agents outside of
> metering.
>
> If an agent is to be built it is a not insignificant change in
> architecture for openstack.


This agent will run on the compute host, next to nova-compute, not inside
the VM. Is that still a concern?


>
>
> -Matt
>
>
> On Tue, May 22, 2012 at 7:30 AM, Doug Hellmann <
> doug.hellmann@xxxxxxxxxxxxx> wrote:
>
>>
>>
>> On Tue, May 22, 2012 at 4:05 AM, Endre Karlson <endre.karlson@xxxxxxxxx>wrote:
>>
>>> If I'm understanding this correctly, the Collector is kind of like a
>>> Agent in Qantum (It sits on a machine doing stuff and passing info
>>> upstream).
>>>
>>> If you look at the approach they have now in Quantum Agent it's writing
>>> directly to the DB. But looking at the next version they seem to be moving
>>> to having the Agent send data upstream to the Plugin in Quantum. Why not do
>>> something similar?
>>>
>>> I mean if you have a MQ cluster in a deployment I think it makes more
>>> sense to have 1 thing that handles the db stuff then having each Collector
>>> connect to the db..
>>>
>>
>> That was the goal, but I may have swapped the terminology around. For
>> ceilometer, the "agent" runs on the compute node and writes only to the
>> message queue. The "collector" runs in a central location and writes to the
>> database. The number of collectors you need will depend on the number of
>> messages being generated, but the architecture supports running several in
>> parallel in a way that each instance does not need to be aware of the
>> others.
>>
>> Doug
>>
>>
>>> Endre.
>>> 2012/5/22 Nick Barcet <nick.barcet@xxxxxxxxxxxxx>
>>>
>>>> On 05/21/2012 10:52 PM, Doug Hellmann wrote:
>>>> > I have written up some of my thoughts on a proposed design for
>>>> > ceilometer in the wiki [1]. I'm sure there are missing details, but I
>>>> > wanted to start getting ideas into writing so they could be discussed
>>>> > here on the list, since I've talked about different parts with a
>>>> couple
>>>> > of you separately.
>>>> >
>>>> > Let me know what you think, and especially if I am not clear or have
>>>> > left out any details.
>>>> >
>>>> > Thanks,
>>>> > Doug
>>>> >
>>>> > [1]
>>>> http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1
>>>>
>>>> Thanks a lot for putting this together Doug.
>>>>
>>>> A few questions:
>>>>
>>>> * "The collector runs on one or more central management servers to
>>>> monitor the message queues (for notifications and for metering data
>>>> coming from the agent). Notification messages are processed and turned
>>>> into metering messages and sent back out onto the message bus using the
>>>> appropriate topic. Metering messages are written to the data store
>>>> without modification."
>>>> -> Is the reason behind why collectors do not write directly to the
>>>> database a way to allow db less implementations as Francis suggested
>>>> earlier?  In this case it may be useful to say it explicitly.
>>>>
>>>> * "Plugins may require configuration options, so when the plugin is
>>>> loaded it is asked to add options to the global flags object, and the
>>>> results are made available to the plugin before it is asked to do any
>>>> work."
>>>> -> I am not sure where the "global flags object" resides and how option
>>>> are populated.  I think it would make sense for this to be globally
>>>> controlled, and therefore may require for a simple discovery exchange on
>>>> the queue to retrieve values and set defaults if it does not exist yet.
>>>>
>>>> * "Metering messages are signed using the hmac module in Python's
>>>> standard library. A shared secret value can be provided in the
>>>> ceilometer configuration settings. The messages are signed by feeding
>>>> the message key names and values into the signature generator in sorted
>>>> order. Non-string values are converted to unicode and then encoded as
>>>> UTF-8. The message signature is included in the message for verification
>>>> by the collector."
>>>> -> The signature is also kept in the database for future audit
>>>> processes, maybe worth mentioning it here.
>>>> -> In addition to a signature, I think we would need a sequence number
>>>> to be embedded by the agent for each message sent, so that loss of
>>>> messages, or forgery of messages, can be detected by the collector and
>>>> further audit process.
>>>>
>>>> Thanks again,
>>>> Nick
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>