openstack team mailing list archive

[Zhongyue Luo <lzyeval@xxxxxxxxx>]

Brian,

Dough isn't dependent on Horizon.

Dough has a client that can be inserted anywhere to notify the start/end of
using a resource.

We inserted the client in horizon just to try out the many ways our billing
system could be integrated with our existing deployment.

We are planning to make a Nova notifier to listen on the AMQP to automate
the subscription process.

However, we still have to devise a way to integrate Swift that would
be symmetric with the things done in Nova.

Also fail recovery is a feature we are working on right now. Separate the
log and execution to cross-reference the logs and DB periodically.

We plan to finish the first release this month and use it on our deployment.

Let me know if you have other suggestions.

Cheers,
LZY

On Tue, Apr 24, 2012 at 1:06 AM, Doug Hellmann
<doug.hellmann@xxxxxxxxxxxxx>wrote:

>
>
> On Sun, Apr 22, 2012 at 5:29 PM, Brian Schott <
> brian.schott@xxxxxxxxxxxxxxxxxx> wrote:
>
>> Dough is a proposed billing service.  There was a session at Folsom
>> design summit.  This is a practical project for an OpenStack provider with
>> test code on github.
>> http://summit.openstack.org/sessions/view/37
>>
>> "
>> The billing system consists of three components.
>> 1) API Server, which receives subscribe/unsubscribe and usage query
>> requests.
>> 2) Farmer, which periodically checks all subscriptions and requests
>> billing to the collector.
>> 3) Collector, gets work from the farmer and uses python-novaclient,
>> kanyun-client, swift-client to retrieve usage information of a product the
>> tenant has subscribed and charges money to the accordingly.
>> "
>> Kanyun is an OpenStack monitoring tool, but I don't see documentation in
>> the code tree (but I've just pulled it to see).
>> https://github.com/lzyeval
>>
>> Dough had mixed reviews during the session, but that I think was because
>> it came across as a billing system triggered solely by Horizon requests and
>> was a polled architecture.
>>
>
> I also had some reservations about Dough because it seemed brittle. For
> example, it was my impression that if an instance creation operation failed
> for some reason, the operator had to manually correct the billing data.
>
>
>> Given what we have today, I'd implement my own billing/metering system
>> exactly the same way.  In fact, I have.  Our own e-commerce system at
>> Nimbis works this way with EC2 and OpenStack presently because the only
>> option available is polling periodically and logging the usage data.
>>
>
> The fact that it only worked through Horizon was the biggest issue for us.
> That means if we expose the API to our users, instances created via the API
> but not the Horizon dashboard would not be billed. I understood the
> decision to be a proof-of-concept, but an event-based model for collecting
> the data seems much more robust and flexible.
>
>
>> Where I'd like to see OpenStack go is metering service that is fully
>> asynchronous and event driven so that I only need to hit the API service
>> when I'm generating an invoice, rather than once per minute because I don't
>> know when an instance was terminated by the user or just crashed.
>>
>> -------------------------------------------------
>> Brian Schott, CTO
>> Nimbis Services, Inc.
>> brian.schott@xxxxxxxxxxxxxxxxxx
>> ph: 443-274-6064  fx: 443-274-6060
>>
>>
>>
>> On Apr 22, 2012, at 4:57 PM, Endre Karlson wrote:
>>
>> What is Dough then compared to what you want to do ?
>>
>> 2012/4/22 Endre Karlson <endre.karlson@xxxxxxxxx>
>>
>>> What is Dough then ?
>>>
>>>
>>> 2012/4/22 Brian Schott <brian.schott@xxxxxxxxxxxxxxxxxx>
>>>
>>>> I see this blueprint for metering, but none for Dough currently.
>>>> http://wiki.openstack.org/EfficientMetering
>>>>
>>>> Here are the Dough slides, however:
>>>> http://www.slideshare.net/lzyeval/dough-openstack-billing-project
>>>>
>>>> We collectively need to talk more about the user scenarios, because I
>>>> don't think you can just put a decorator around the API rpc calls and get
>>>> an accurate picture of what to bill for later.  There are metered things
>>>> like bandwidth or IOPS, events that happen outside of the API (shutdown
>>>> -h), and it is hard to predict what a reseller will want to charge for.  It
>>>> is better to put a metering system in that can handle many billing
>>>> configurations.
>>>>
>>>>
>>>> -------------------------------------------------
>>>> Brian Schott, CTO
>>>> Nimbis Services, Inc.
>>>> brian.schott@xxxxxxxxxxxxxxxxxx
>>>> ph: 443-274-6064  fx: 443-274-6060
>>>>
>>>>
>>>>
>>>> On Apr 22, 2012, at 3:36 PM, Luis Gervaso wrote:
>>>>
>>>> Dough is the proposed billing platform/product (where the billing rules
>>>> live), isn't it?
>>>>
>>>> I don't know Dough enough, so please me correct me if i'm wrong.
>>>>
>>>> I'm trying to define a generic/agnostic integration process, obviously
>>>> where Dough
>>>> can fit perfectly. I would like it become part to the reference
>>>> architecture.
>>>>
>>>> Option 1) [3b in the arch proposed]
>>>>
>>>> Dough should pull NoSQL
>>>>
>>>> Option 2)
>>>>
>>>> A Mediator can feed Dough
>>>>
>>>>
>>>> On Sun, Apr 22, 2012 at 9:13 PM, Endre Karlson <endre.karlson@xxxxxxxxx
>>>> > wrote:
>>>>
>>>>> What about using the Dough project?
>>>>>
>>>>> Endre.
>>>>>
>>>>>
>>>>> 2012/4/22 Endre Karlson <endre.karlson@xxxxxxxxx>
>>>>>
>>>>>> What about using the Dough project ?
>>>>>>
>>>>>> Endre.
>>>>>>
>>>>>> 2012/4/22 Luis Gervaso <luis@xxxxxxxxx>
>>>>>>
>>>>>>>  Hi,
>>>>>>>
>>>>>>> I want to share the architecture i am developing in order to perform
>>>>>>> the monitorig / billing OpenStack support:
>>>>>>>
>>>>>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
>>>>>>> interchangeable) (Own Stuff or ServiceMix / Camel)
>>>>>>>
>>>>>>> 2. Events should be stored on a NoSQL document oriented database (I
>>>>>>> think mongodb is perfect, since we can query in a super easy fashion)
>>>>>>>
>>>>>>> 3a. The monitoring system can pull/push MongoDB
>>>>>>>
>>>>>>> 3b. The billing system can pull to create invoices
>>>>>>>
>>>>>>> 4. A mediation EIP should be necessary to integrate a
>>>>>>> billing/monitoring product. (ServiceMix / Camel)
>>>>>>>
>>>>>>>  This is to receive your feedback. So please, critics are welcome!
>>>>>>>
>>>>>>> Cheers!
>>>>>>>
>>>>>>> --
>>>>>>> -------------------------------------------
>>>>>>> Luis Alberto Gervaso Martin
>>>>>>> Woorea Solutions, S.L
>>>>>>> CEO & CTO
>>>>>>> mobile: (+34) 627983344
>>>>>>> luis@ <luis.gervaso@xxxxxxxxx>woorea.es
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~openstack
>>>>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~openstack
>>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> -------------------------------------------
>>>> Luis Alberto Gervaso Martin
>>>> Woorea Solutions, S.L
>>>> CEO & CTO
>>>> mobile: (+34) 627983344
>>>> luis@ <luis.gervaso@xxxxxxxxx>woorea.es
>>>>
>>>>  _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>>
>>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>