openstack team mailing list archive

[Matt Dietz <matt.dietz@xxxxxxxxxxxxx>]

If you wanted to use Yagi, it would be trivial to add a JSON only notifier to Yagi. If you're interested and need a hand, feel free to hit Dragon or I for assistance.

From: Monsyne Dragon <mdragon@xxxxxxxxxxxxx<mailto:mdragon@xxxxxxxxxxxxx>>
Date: Mon, 23 Apr 2012 16:39:08 +0000
To: Luis Gervaso <luis@xxxxxxxxx<mailto:luis@xxxxxxxxx>>
Cc: "<openstack@xxxxxxxxxxxxxxxxxxx<mailto:openstack@xxxxxxxxxxxxxxxxxxx>>" <openstack@xxxxxxxxxxxxxxxxxxx<mailto:openstack@xxxxxxxxxxxxxxxxxxx>>
Subject: Re: [Openstack] Monitoring / Billing Architecture proposed

This already exists in trunk.  The Notification system was designed specifically to feed billing and monitoring systems.

Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone,  so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules.

On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:

Hi,

I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:

1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)

2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)

We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database.  It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol.

That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )




3a. The monitoring system can pull/push MongoDB

3b. The billing system can pull to create invoices

4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)

This is to receive your feedback. So please, critics are welcome!

Cheers!

--
-------------------------------------------
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO & CTO
mobile: (+34) 627983344
luis@<mailto:luis.gervaso@xxxxxxxxx>woorea.es<http://woorea.es/>


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx<mailto:openstack@xxxxxxxxxxxxxxxxxxx>
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

--
Monsyne M. Dragon
OpenStack/Nova
cell 210-441-0965
work x 5014190

_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@xxxxxxxxxxxxxxxxxxx<mailto:openstack@xxxxxxxxxxxxxxxxxxx> Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp