openstack team mailing list archive

[Doug Hellmann <doug.hellmann@xxxxxxxxxxxxx>]

On Fri, May 11, 2012 at 3:40 PM, Loic Dachary <loic@xxxxxxxxxxxx> wrote:

> On 05/11/2012 05:55 PM, Doug Hellmann wrote:
> > During yesterday's meeting we discussed the API proposal at
> > http://wiki.openstack.org/EfficientMetering#API and came up with a few
> > missing items and other points for discussion. We should try to work out
> > those details on the list before the meeting next week.
> >
> > The original proposal included these API calls:
> >
> > GET list components
> > GET list components meters (argument : name of the component)
> > GET list [user_id|project_id|source]
> > GET list of meter_type
> > GET list of events per [user_id|project_id|source] ( allow to specify
> user_id or project_id or both )
> > GET sum of (meter_volume, meter_duration) for meter_type and
> [user_id|project_id|source]
> >
> > They can be broken down into three groups:
> >
> > - Discover the sorts of things the server can provide:
> >
> > - list the components providing metering data
> > - list the meters for a given component
> > - list the known users
> > - list the known projects
> > - list the known sources
> > - list the types of meters known
> >
> > - Fetch raw event data, without aggregation:
> >
> > - per user
> > - per project
> > - per source
> > - per user and project
> >
> > - Produce aggregate views of the data:
> >
> > - sum "volume" field for meter type over a time period
> > - per user
> > - per project
> > - per source
> > - sum "duration" field for meter type over a period of time
> > - per user
> > - per project
> > - per source
> I updated the wiki with these three groups, it is a good starting point
> for discussion.
> http://wiki.openstack.org/EfficientMetering?action=diff&rev2=75&rev1=74
> >
> > We agreed that all of the queries related to metering data would need
> > to have parameters to set the start and end times with the start time
> > inclusive and the end time exclusive (i.e., start <= timestamp < end).
> >
> > Callers also are likely to want to filter the raw data queries based
> > on the meter type, so we need to add that as an optional argument
> > there. The aggregate values have the meter type as a required argument
> > (because it does not make sense to aggregate data from separate meters
> > together).
> >
> > There are a few other queries missing from that list. The items below
> > are based on the meeting notes and my own thoughts from yesterday,
> > please add to the list if I have left anything out (or suggest why we
> > should not implement something suggested here, of course):
> >
> > - list discrete events that may not have a duration (instance
> > creation, IP allocation, etc.)
> > - list raw event data for a resource ("what happened with a specific
> > instance?")
> > - aggregate event data per meter type for a resource over a period of
> > time ("what costs are related to this instance?")
> > - sum volume for meter type over a time period for a specific resource
> > ("how much total bandwidth was used by a VIF?")
> > - sum duration for meter type over a time period for a specific
> > resource ("how long did an instance run?")
> > - metadata for resources (such as location of instances)
> > - aggregating averages in addition to sums
> >
> I've added this list as a note in the API chapter.
> http://wiki.openstack.org/EfficientMetering?action=diff&rev2=76&rev1=75
> > Some of these items may be better handled in the consumer of this API
> > (the system that actually bills the user). I am curious to know how
> > others are handling auditing or other billing detail reporting for
> > users.
> I will be in a meeting with the participants of the
> http://www.opencloudware.org/ project early next week and I'll ask them.
> Maybe Daniel Dyer could share a practical experience with us.
>
> > In order to support the discrete events, we need to capture
> > them. Perhaps those could be implemented as additional counter
> > types. Thoughts?
> Are there resources that should be billed (henced metered) if their
> lifespan is shorter than the polling / publishing event period ? I mean
> that if someone creates a VM and immediately destroys it, does it really
> matter. While writing this statement, I though of a way to abuse a system
> that does not account for discrete events like "created a VM" but only
> account for "this VM has run for at least X minutes" :-) One could write a
> software to launch short lived VM at specific times and get computing power
> for free.
>

Exactly. We may bill the customer for the first X minutes of use for a VM
just for creating it, and then do the math necessary to avoid double
billing them during the first billing cycle.

Do discrete event need to be separate ? Back to the example above, someone
> creates a VM : that does not create a record in the ceilometer storage. The
> VM is destroyed a few seconds later : it creates an event, as if it was
> polled / published to measure the uptime of the VM. The c1 meter would show
> the VM has been up for 15 seconds since it was created. If the c1 meter is
> generated from an exist event captured from
> http://wiki.openstack.org/SystemUsageData that occurs every 5 minutes,
> the uptime will be 5 minutes more than the previous one.
>

VMs may not be the best example for discussing discrete events since so
much of what happens there is related to time. A better example is creating
images. We may charge the user a one-time fee for uploading an image to
Glance, and then charge an ongoing fee for the storage consumed. So the
one-time fee is a discrete event based on creating the image.


>
> While thinking about this use case I also had a simple idea to optimize
> the storage of events and added it to the wiki
> http://wiki.openstack.org/EfficientMetering?action=diff&rev2=77&rev1=76
>
> Cheers
>
> --
> Loïc Dachary         Chief Research Officer
> // eNovance labs   http://labs.enovance.com
> // ✉ loic@xxxxxxxxxxxx  ☎ +33 1 49 70 99 82
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>