openstack team mailing list archive

[Loic Dachary <loic@xxxxxxxxxxxx>]

On 05/16/2012 09:32 AM, Nick Barcet wrote:
>> On Tue, May 15, 2012 at 10:26 AM, Doug Hellmann
>> <doug.hellmann@xxxxxxxxxxxxx <mailto:doug.hellmann@xxxxxxxxxxxxx>> wrote:
>>
>>
>>
>>     On Tue, May 15, 2012 at 8:21 AM, Julien Danjou
>>     <julien.danjou@xxxxxxxxxxxx <mailto:julien.danjou@xxxxxxxxxxxx>> wrote:
>>
>>         On Tue, May 15 2012, Loic Dachary wrote:
>>
>>         > On 05/15/2012 12:05 PM, Julien Danjou wrote:
>>         >>
>>         >> OTOH I find the metadata proposal in another table too much
>>         >> complicated. Why not storing what metadata in the
>>         meter.payload field
>>         >> in the same table (e.g. as a JSON string)?
>>         > I would be much simpler to store the metadata in the
>>         resource_id field
>>         > which could be renamed into resource field.
>>
>>         That'd be even more radical.
>>
>>
>>     I like it because it would simplify the messaging. We can leave the
>>     storage optimization question to the daemon that stores the data.
>>
>>         > Instead of resource_id=134123 we could have resource={ 'id':
>>         134123,
>>         > 'name': 'foobar', 'flavor': 'm1.small' etc.. } There would be
>>         no need
>>         > for versioning, format, separate table, etc. etc. The only
>>         convention
>>         > would be that it's a hash with at least one field : the id of the
>>         > resource. The rest is metadata.
>>         >
>>         > It will use a lot of disk space with highly redundant information.
>>
>>         Ok, so the current proposal is just early optimization, as I
>>         understood.
>>
>>         If you want to optimize the storage, why not use resource_id as a
>>         foreign key to the metatable table which would contains unique
>>         records
>>         of metadata?
>>
>>         That would allow to store identical metadata once (and therefore
>>         optimize space) and will be much simpler. There would not be any
>>         need of
>>         version, timestamp, or whatever on metadata.
> I fully second Julien's analysis regarding the storage of metadata in a
> separate table.  We are overly complicating the schema for the sole
> purpose of resource optimization.  In fact, the current metadata table
> seems to be defining 5 fields where only one constitutes some valid
> content we would like to extract at some point, the rest existing only
> for relational or versioning purposes.  A bit of an overkill, it seems.
>
> I would not recommend however to overcharge a field that can be used as
> a key value for searches (resource_id) to store additional data, as it
> would block us to search on that key easily.  Why mot just extend the
> meter schema to add a metadata field, which can use the proposed JSON
> format?
>
It makes sense and I updated the wiki accordingly:

http://wiki.openstack.org/EfficientMetering?action=diff&rev2=81&rev1=80

What do you think ?

-- 
Loïc Dachary         Chief Research Officer
// eNovance labs   http://labs.enovance.com
// ✉ loic@xxxxxxxxxxxx  ☎ +33 1 49 70 99 82