openstack team mailing list archive

[Sandy Walsh <sandy.walsh@xxxxxxxxxxxxx>]

> Thanks for putting that together Sandy. Very nice!

Thanks!

>From my perspective, there are two major things that are undesirable for us:
>
>1) Putting this data through the queue is not something that feels right. We'd like to have to option to use other types of data "sinks" from the generation points. Some folks might want to use the queues, but we do not wish to burden the queueing system with this volume of data. In some cases, we will just drop these into log files for later collection and aggregation.

Makes sense. You mentioned at the summit about json structured log files, which would help. But additionally the existing notifier driver can easily be used to write to
   - a file
   - the logs (I think one exists)
   - a different rabbit queue (something we're considering)

so it need never hit the production rabbit.

We'd need a different worker mechanism for parsing the log files (keeping track of what's been done, etc). It might be hard to horizontally scale that though. Also, I'm not sure if latency of writing the log over the network or aggregating the local log files would be any different than writing to another rabbit (which is largely in memory). 

I'll update the doc to reflect this.

(I assume we're talking monitoring here, I would never put instrumentation stuff in the queues)

> 2) We would like a common mechanism for instrumenting but we would like to be able to route data to any number of places: local file, datagram endpoint, etc.

Yup ... Tach has a driver based notifier mechanism. Easy to do.

>Now, getting the lower level measurement library consistent is definitely the right approach. I still think we need to support decorators in addition to monkey patching. And, we should make the gauges or whatever we call them usable with different "sinks".

Hmm, really not a fan of the decorator approach. Makes the code really ugly. They'd be everywhere. 

Not sure if I can get over it. :D

-S

On Nov 1, 2012, at 1:17 PM, Sandy Walsh wrote:

> Hey!
>
> Here's a first pass at a proposal for unifying StackTach/Ceilometer and other instrumentation/metering/monitoring efforts.
>
> It's v1, so bend, spindle, mutilate as needed ... but send feedback!
>
> http://wiki.openstack.org/UnifiedInstrumentationMetering
>
> Thanks,
> Sandy
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp