openstack team mailing list archive

[Nick Barcet <nick.barcet@xxxxxxxxxxxxx>]

On 04/23/2012 10:45 PM, Doug Hellmann wrote:
> 
> 
> On Mon, Apr 23, 2012 at 4:14 PM, Brian Schott
> <brian.schott@xxxxxxxxxxxxxxxxxx
> <mailto:brian.schott@xxxxxxxxxxxxxxxxxx>> wrote:
> 
>     Doug,
> 
>     Do we mirror the table structure of nova, etc. and add
>     created/modified columns? 
> 
> 
>     Or do we flatten into an instance event record with everything?  
> 
> 
> I lean towards flattening the data as it is recorded and making a second
> pass during the bill calculation. You need to record instance
> modifications separately from the creation, especially if the
> modification changes the billing rate. So you might have records for:
> 
> created instance, with UUID, name, size, timestamp, ownership
> information, etc.
> resized instance, with UUID, name, new size, timestamp, ownership
> information, etc.
> deleted instance, with UUID, name, size, timestamp, ownership
> information, etc.
> 
> Maybe some of those values don't need to be reported in some cases, but
> if you record a complete picture of the state of the instance then the
> code that aggregates the event records to produce billing information
> can use it to make decisions about how to record the charges.
> 
> There is also the case where an instance is still no longer running but
> nova thinks it is (or the reverse), so some sort of auditing sweep needs
> to be included (I think that's what Dough called the "farmer" but I
> don't have my notes in front of me).

When I wrote [1], one of the things that I never assumed was how agents
would collect their information. I imagined that the system should allow
for multiple implementation of agents that would collect the same
counters, assuming that 2 implementations for the same counter should
never be running at once.

That said, I am not sure an event based collection of what nova is
notifying would satisfy the requirements I have heard from many cloud
providers:
- how do we ensure that event are not forged or lost in the current nova
system?
- how can I be sure that an instance has not simply crashed and never
started?
- how can I collect information which is not captured by nova events?

Hence the proposal to use a dedicated event queue for billing, allowing
for agents to collect and eventually validate data from different
sources, including, but not necessarily limiting, collection from the
nova events.

Moreover, as soon as you generalize the problem to other components than
just Nova (swift, glance, quantum, daas, ...) just using the nova event
queue is not an option anymore.

[1] http://wiki.openstack.org/EfficientMetering

Nick

Attachment: signature.asc
Description: OpenPGP digital signature