openstack team mailing list archive

[Doug Hellmann <doug.hellmann@xxxxxxxxxxxxx>]

On Mon, Apr 30, 2012 at 11:43 AM, Loic Dachary <loic@xxxxxxxxxxxx> wrote:

>  On 04/30/2012 03:49 PM, Doug Hellmann wrote:
>
>
>
> On Mon, Apr 30, 2012 at 6:46 AM, Loic Dachary <loic@xxxxxxxxxxxx> wrote:
>
>> On 04/30/2012 12:15 PM, Loic Dachary wrote:
>> > We could start a discussion from the content of the following sections:
>> >
>> > http://wiki.openstack.org/EfficientMetering#Counters
>>  I think the rationale of the counter aggregation needs to be explained.
>> My understanding is that the metering system will be able to deliver the
>> following information: 10 floating IPv4 addresses were allocated to the
>> tenant during three months and were leased from provider NNN. From this,
>> the billing system could add a line to the invoice : 10 IPv4, $N each =
>> $10xN because it has been configured to invoice each IPv4 leased from
>> provider NNN for $N.
>>
>> It is not the purpose of the metering system to display each IPv4 used,
>> therefore it only exposes the aggregated information. The counters define
>> how the information should be aggregated. If the idea was to expose each
>> resource usage individually, defining counters would be meaningless as they
>> would duplicate the activity log from each OpenStack component.
>>
>> What do you think ?
>>
>
>  At DreamHost we are going to want to show each individual resource (the
> IPv4 address, the instance, etc.) along with the charge information. Having
> the metering system aggregate that data will make it difficult/impossible
> to present the bill summary and detail views that we want. It would be much
> more useful for us if it tracked the usage details for each resource, and
> let us aggregate the data ourselves.
>
>  If other vendors want to show the data differently, perhaps we should
> provide separate APIs for retrieving the detailed and aggregate data.
>
>  Doug
>
>   Hi,
>
> For the record, here is the unfinished conversation we had on IRC
>
> (04:29:06 PM) dhellmann: dachary, did you see my reply about counter
> definitions on the list today?
> (04:39:05 PM) dachary: It means some counters must not be aggregated.
> Only the amount associated with it is but there is one counter per IP.
> (04:55:01 PM) dachary: dhellmann: what about this :the id of the
> ressource controls the agregation of all counters : if it is missing, all
> resources of the same kind and their measures are aggregated. Otherwise
> only the measures are agreggated.
> http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39
> (04:55:58 PM) dachary: it makes me a little unconfortable to define such
> an "ad-hoc" grouping
> (04:56:53 PM) dachary: i.e. you actuall control the aggregation by
> chosing which value to put in the id column
> (04:58:43 PM) dachary: s/actuall/actually/
> (05:05:38 PM) ***dachary reading http://www.ogf.org/documents/GFD.98.pdf
> (05:05:54 PM) dachary: I feel like we're trying to resolve a non problem
> here
> (05:08:42 PM) dachary: values need to be aggregated. The raw input is a
> full description of the resource and a value ( gauge ). The question is how
> to control the aggregation in a reasonably flexible way.
> (05:11:34 PM) dachary: The definition of a counter could probably be
> described as : the id of a resource and code to fill each column associated
> with it.
>
> I tried to append the following, but the wiki kept failing.
>
> Propose that the counters are defined by a function instead of being
> fixed. That helps addressing the issue of aggregating the bandwidth
> associated to a given IP into a single counter.
>
> Alternate idea :
>  * a counter is defined by
>   * a name ( o1, n2, etc. ) that uniquely identifies the nature of the
> measure ( outbound internet transit, amount of RAM, etc. )
>   * the component in which it can be found ( nova, swift etc.)
>  * and by columns, each one is set with the result of
> aggregate(find(record),record) where
>   * find() looks for the existing column as found by selecting with the
> unique key ( maybe the name and the resource id )
>   * record is a detailed description of the metering event to be
> aggregated (
> http://wiki.openstack.org/SystemUsageData#compute.instance.exists: )
>   * the aggregate() function returns the updated row. By default it just
> += the counter value with the old row returned by find()
>

Would we want aggregation to occur within the database where we are
collecting events, or should that move somewhere else?


>
>
> Cheers
>
> --
> Loïc Dachary         Chief Research Officer
> // eNovance labs   http://labs.enovance.com
> // ✉ loic@xxxxxxxxxxxx  ☎ +33 1 49 70 99 82
>
>