openstack team mailing list archive

[Joshua Harlow <harlowja@xxxxxxxxxxxxx>]

I like http://wiki.openstack.org/SystemUsageData

I just wonder if there should be offical formats for the data sent across the wire.

Or at least high level formats that define the data, maybe in a simple python "map" style layout.

With examples would be awesome.

On 4/23/12 10:40 AM, "Luis Gervaso" <luis@xxxxxxxxx> wrote:

I have been looking at : http://wiki.openstack.org/SystemUsageData

On Mon, Apr 23, 2012 at 7:35 PM, Brian Schott <brian.schott@xxxxxxxxxxxxxxxxxx> wrote:
Is there a document somewhere on what events the services emit?

-------------------------------------------------
Brian Schott, CTO
Nimbis Services, Inc.
brian.schott@xxxxxxxxxxxxxxxxxx
ph: 443-274-6064 <tel:443-274-6064>   fx: 443-274-6060 <tel:443-274-6060>



On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:

This already exists in trunk.  The Notification system was designed specifically to feed billing and monitoring systems.

Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone,  so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules.

On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:

Hi,

I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:

1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)

2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)

We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database.  It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol.

That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )




3a. The monitoring system can pull/push MongoDB

3b. The billing system can pull to create invoices

4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)

This is to receive your feedback. So please, critics are welcome!

Cheers!