openstack team mailing list archive

[Alan Sill <Alan.Sill@xxxxxxx>]

The following links to work from OGF in the usage accounting and tracking area might be useful.  

First of all, we have the UsageRecord format (UR), which has been used in a variety of distributed computing environments for tracking usage.  It defines an XML-based format for exchange of usage records.  The format is general enough to describe a variety of distributed computing environments, but specific enough to allow practical usage and has served as the basis for usage accounting in grids, for example, for some time.

  http://ogf.org/documents/GFD.98.pdf

We also have the following storage accounting work in process and nearly complete:

https://forge.gridforum.org/sf/go/artf6532?nav=1 and
http://www.ogf.org/Public_Comment_Docs/Documents/2012-02/EMI-StAR-OGF-info-doc-v2.pdf 

This document has just come out of public comment and is moving toward publication shortly.

Between the UR work on traditional cpu accounting and the above storage accounting publication, I hope this provides some input into this topic from OGF.

Alan

On Apr 23, 2012, at 12:50 PM, Brian Schott wrote:

> So, we could build on this. No reason to reinvent, but we might want to expand the number of events.  I'm concerned about things like what happens when flavors change over time.  Maybe the answer is, always append to the flavor/instance-type table.  The code I remember and the admin interface that Ken wrote allowed you to modify flavors.  That would break billing unless you also track flavor modifications.
> 
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@xxxxxxxxxxxxxxxxxx
> ph: 443-274-6064  fx: 443-274-6060
> 
> 
> 
> On Apr 23, 2012, at 1:40 PM, Luis Gervaso wrote:
> 
>> I have been looking at : http://wiki.openstack.org/SystemUsageData
>> 
>> On Mon, Apr 23, 2012 at 7:35 PM, Brian Schott <brian.schott@xxxxxxxxxxxxxxxxxx> wrote:
>> Is there a document somewhere on what events the services emit?  
>> 
>> -------------------------------------------------
>> Brian Schott, CTO
>> Nimbis Services, Inc.
>> brian.schott@xxxxxxxxxxxxxxxxxx
>> ph: 443-274-6064  fx: 443-274-6060
>> 
>> 
>> 
>> On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:
>> 
>>> This already exists in trunk.  The Notification system was designed specifically to feed billing and monitoring systems. 
>>> 
>>> Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone,  so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules. 
>>> 
>>> On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:
>>> 
>>>> Hi,
>>>> 
>>>> I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:
>>>> 
>>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)
>>>  
>>>> 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)
>>> 
>>> We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database.  It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol. 
>>> 
>>> That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )
>>> 
>>> 
>>> 
>>>> 
>>>> 3a. The monitoring system can pull/push MongoDB
>>>> 
>>>> 3b. The billing system can pull to create invoices 
>>>> 
>>>> 4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)
>>>> 
>>>> This is to receive your feedback. So please, critics are welcome!
>>>> 
>>>> Cheers!
>>>> 
>>>> -- 
>>>> -------------------------------------------
>>>> Luis Alberto Gervaso Martin
>>>> Woorea Solutions, S.L
>>>> CEO & CTO
>>>> mobile: (+34) 627983344
>>>> luis@xxxxxxxxx
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help   : https://help.launchpad.net/ListHelp
>>> 
>>> --
>>> Monsyne M. Dragon
>>> OpenStack/Nova 
>>> cell 210-441-0965
>>> work x 5014190
>>> 
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>> 
>> 
>> 
>> 
>> -- 
>> -------------------------------------------
>> Luis Alberto Gervaso Martin
>> Woorea Solutions, S.L
>> CEO & CTO
>> mobile: (+34) 627983344
>> luis@xxxxxxxxx
>> 
> 
> <smime.p7s>_______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp