← Back to team overview

openstack team mailing list archive

Re: [metering] Where can I consume messages from metering system?

 

On 25/05/12 10:55 +0200, Szymon Grzybowski wrote:
Hi,


Hi Szymon,

I have very simerlar needs, see comments inline...

(I am working on the heat project "https://github.com/heat-api/heat";)


We had a plan to write some "plugin" which will collect metering data from
VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts)
via libvirt (mayby later for xen etc), but i've found ceilometer project,
which is doing what I want or will be doing what I want in near future :).
I was thinking about collectd/ganglia/munin as source of data and I wanted
to write local agents which will poll those daemons (collectd/ganglia) and
push data to Rabbit, so it's pretty close to ceilometer's guys.

My main purpose is to consume those messages and write algorithm to manage
cloud from administrative perspective. I'd like to create something like
DRM in VmWare - I was on presentation about this topic recently and from
that brief I learnt that VmWare has something like rules to manage VMs. I
think, that example (use case) from presentation would be the best way to
describe what I'm trying to point out:

1. Administrator defined a rule: { If free ram on host is less than 15%,
send notification "Alert less than 15% free ram on host"}
2. HostA hits less than 15% of free ram and sends notification "Alert less
than 15% free ram on HostA".
3. Central collector or central daemon receives Alert about free ram.
4. Central collector pick VM from HostA and migrate it to HostB. - How he
decides which VM and to which Host migrate is part of algorithm/policy.

I am trying to implement AWS::CloudWatch::Alarm
http://docs.amazonwebservices.com/AWSCloudFormation/latest/UserGuide/aws-properties-cw-alarm.html

This defines really simerlar rules to what you are describing, and also
allows users to post (rest) statistics back to a statistics Collecting
server. We could then do fun things like autoscaling and HA recovery.



This is the basic idea, how does it work - this is what I've found out
during presentation. And now my whole idea about this mechanism and
openstack: there could be different policies(algorithms/plugins) how, where
and when migrate VMs and this could be implemented in external plugins.

Someone can say that there is nova-scheduler. This scenario isn't exactly
what nova-scheduler is doing, but I see, that in this solution
nova-scheduler can play his role as service, which picks best host to
migrate VM.

I'm trying to find out what can I use from ceilometer as metering system.
I've read few pages on wiki and posts on mailing list about ceilometer's
architecture, messages etc.
Right now I know there are Agents and Collectors. Agents get data from
hypervisor (metering data)  and push them in queue (nova notification
system). Collectors listens to queue's topic and receives those messages.
I'd like to:

  - write plugin for agent to send proper alert when metering data hits
  proper rules (just like in above scenario with 15% free ram)
  - write service which will consume this alert (listen proper topic in
  queue) and fetch data from ceilometer collector?? (or it's backend via API)
  and find out which VM should be migrated and where (or delegate "Where" to
  nova-scheduler)

I could help out with work on rules and receiving guest statistics if
ceilometer guys are ok with this extension.


Another idea. Are there any Host states in Openstack? For example
"Maintenance state"? Besides scenarios with free ram, cpu and other
resources there is also typical example of how useful this mechanism could
be with maintenance state.
*Scenario:*
1. Administrator changes HostA state to "maintenance".
2. There goes alert "HostA maintenance".
3. Central collector or central daemon receives Alert about maintenance.
4. Central collector gets list of VMs on HostA.
5. Central collectors start migration all VMs from HostA to other Hosts (or
invoke nova-scheduler to reassign VM from HostA to other hosts, but in this
case we need to change or implement new filter in nova-scheduler)
6. Administrator changes HostA state to "active".
7. Openstack starts new machines on this node (this is now done via
nova-scheduler) or migrate VMs from other overloaded hosts to HostA.

Everything should be event-driven (alert-notification, host-state-change,
administrator's action).

I'm also wondering what about this
ResourceMonitorAlertsandNotifications<http://wiki.openstack.org/ResourceMonitorAlertsandNotifications>,
because when I'm looking at schema, it has everything what I need, but
blueprint link is broken. Is this idea obsolete? Or is it implemented?
(this was created 2nd April 2012, so I think it's not implemented yet).

Me too (I also have been wondering about it), I was quite keen on this
and contacted the authors, but no response...

Regards
Angus Salkeld


First question: Does it make sense? Are such mechanisms implemented in
Openstack right now? Or is it worth to implement?
Second question (if first's answer isn't negative): How and when can I use
ceilometer to do what I've described? From meeting's
topics<http://wiki.openstack.org/Meetings/MeteringAgenda>I know that
yesterday you were deciding what to use as notification bus,
and in june you will be thinking about storage backend, how is it going on
right now? Is such scenario possible in ceilometer (I mean plugins in
agents)? Collecting data not only from guests but also from hosts.

*Cheers,
*
*
*

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



References