← Back to team overview

openstack team mailing list archive

Re: Monitoring / Billing Architecture proposed

 

Dough is a proposed billing service.  There was a session at Folsom design summit.  This is a practical project for an OpenStack provider with test code on github.  
http://summit.openstack.org/sessions/view/37

"
The billing system consists of three components.
1) API Server, which receives subscribe/unsubscribe and usage query requests.
2) Farmer, which periodically checks all subscriptions and requests billing to the collector.
3) Collector, gets work from the farmer and uses python-novaclient, kanyun-client, swift-client to retrieve usage information of a product the tenant has subscribed and charges money to the accordingly.
"
Kanyun is an OpenStack monitoring tool, but I don't see documentation in the code tree (but I've just pulled it to see).
https://github.com/lzyeval

Dough had mixed reviews during the session, but that I think was because it came across as a billing system triggered solely by Horizon requests and was a polled architecture.  Given what we have today, I'd implement my own billing/metering system exactly the same way.  In fact, I have.  Our own e-commerce system at Nimbis works this way with EC2 and OpenStack presently because the only option available is polling periodically and logging the usage data.

Where I'd like to see OpenStack go is metering service that is fully asynchronous and event driven so that I only need to hit the API service when I'm generating an invoice, rather than once per minute because I don't know when an instance was terminated by the user or just crashed.  

-------------------------------------------------
Brian Schott, CTO
Nimbis Services, Inc.
brian.schott@xxxxxxxxxxxxxxxxxx
ph: 443-274-6064  fx: 443-274-6060



On Apr 22, 2012, at 4:57 PM, Endre Karlson wrote:

> What is Dough then compared to what you want to do ?
> 
> 2012/4/22 Endre Karlson <endre.karlson@xxxxxxxxx>
> What is Dough then ?
> 
> 
> 2012/4/22 Brian Schott <brian.schott@xxxxxxxxxxxxxxxxxx>
> I see this blueprint for metering, but none for Dough currently.
> http://wiki.openstack.org/EfficientMetering
> 
> Here are the Dough slides, however:
> http://www.slideshare.net/lzyeval/dough-openstack-billing-project
> 
> We collectively need to talk more about the user scenarios, because I don't think you can just put a decorator around the API rpc calls and get an accurate picture of what to bill for later.  There are metered things like bandwidth or IOPS, events that happen outside of the API (shutdown -h), and it is hard to predict what a reseller will want to charge for.  It is better to put a metering system in that can handle many billing configurations.
> 
> 
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@xxxxxxxxxxxxxxxxxx
> ph: 443-274-6064  fx: 443-274-6060
> 
> 
> 
> On Apr 22, 2012, at 3:36 PM, Luis Gervaso wrote:
> 
>> Dough is the proposed billing platform/product (where the billing rules live), isn't it?
>> 
>> I don't know Dough enough, so please me correct me if i'm wrong.
>> 
>> I'm trying to define a generic/agnostic integration process, obviously where Dough 
>> can fit perfectly. I would like it become part to the reference architecture.
>> 
>> Option 1) [3b in the arch proposed]
>> 
>> Dough should pull NoSQL
>> 
>> Option 2)
>> 
>> A Mediator can feed Dough
>> 
>> 
>> On Sun, Apr 22, 2012 at 9:13 PM, Endre Karlson <endre.karlson@xxxxxxxxx> wrote:
>> What about using the Dough project?
>> 
>> Endre.
>> 
>> 
>> 2012/4/22 Endre Karlson <endre.karlson@xxxxxxxxx>
>> What about using the Dough project ?
>> 
>> Endre.
>> 
>> 2012/4/22 Luis Gervaso <luis@xxxxxxxxx>
>> Hi,
>> 
>> I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:
>> 
>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)
>> 
>> 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)
>> 
>> 3a. The monitoring system can pull/push MongoDB
>> 
>> 3b. The billing system can pull to create invoices 
>> 
>> 4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)
>> 
>> This is to receive your feedback. So please, critics are welcome!
>> 
>> Cheers!
>> 
>> -- 
>> -------------------------------------------
>> Luis Alberto Gervaso Martin
>> Woorea Solutions, S.L
>> CEO & CTO
>> mobile: (+34) 627983344
>> luis@xxxxxxxxx
>> 
>> 
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>> 
>> 
>> 
>> 
>> -- 
>> -------------------------------------------
>> Luis Alberto Gervaso Martin
>> Woorea Solutions, S.L
>> CEO & CTO
>> mobile: (+34) 627983344
>> luis@xxxxxxxxx
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

Attachment: smime.p7s
Description: S/MIME cryptographic signature


Follow ups

References