openstack team mailing list archive

[Angus Salkeld <asalkeld@xxxxxxxxxx>]

On 25/05/12 10:55 +0200, Szymon Grzybowski wrote:
> Hi,

Hi Szymon,

I have very simerlar needs, see comments inline...

(I am working on the heat project "https://github.com/heat-api/heat";)


> We had a plan to write some "plugin" which will collect metering data from
> VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts)
> via libvirt (mayby later for xen etc), but i've found ceilometer project,
> which is doing what I want or will be doing what I want in near future :).
> I was thinking about collectd/ganglia/munin as source of data and I wanted
> to write local agents which will poll those daemons (collectd/ganglia) and
> push data to Rabbit, so it's pretty close to ceilometer's guys.
>
> My main purpose is to consume those messages and write algorithm to manage
> cloud from administrative perspective. I'd like to create something like
> DRM in VmWare - I was on presentation about this topic recently and from
> that brief I learnt that VmWare has something like rules to manage VMs. I
> think, that example (use case) from presentation would be the best way to
> describe what I'm trying to point out:
>
> 1. Administrator defined a rule: { If free ram on host is less than 15%,
> send notification "Alert less than 15% free ram on host"}
> 2. HostA hits less than 15% of free ram and sends notification "Alert less
> than 15% free ram on HostA".
> 3. Central collector or central daemon receives Alert about free ram.
> 4. Central collector pick VM from HostA and migrate it to HostB. - How he
> decides which VM and to which Host migrate is part of algorithm/policy.

I am trying to implement AWS::CloudWatch::Alarm
http://docs.amazonwebservices.com/AWSCloudFormation/latest/UserGuide/aws-properties-cw-alarm.html

This defines really simerlar rules to what you are describing, and also
allows users to post (rest) statistics back to a statistics Collecting
server. We could then do fun things like autoscaling and HA recovery.


> This is the basic idea, how does it work - this is what I've found out
> during presentation. And now my whole idea about this mechanism and
> openstack: there could be different policies(algorithms/plugins) how, where
> and when migrate VMs and this could be implemented in external plugins.
>
> Someone can say that there is nova-scheduler. This scenario isn't exactly
> what nova-scheduler is doing, but I see, that in this solution
> nova-scheduler can play his role as service, which picks best host to
> migrate VM.
>
> I'm trying to find out what can I use from ceilometer as metering system.
> I've read few pages on wiki and posts on mailing list about ceilometer's
> architecture, messages etc.
> Right now I know there are Agents and Collectors. Agents get data from
> hypervisor (metering data)  and push them in queue (nova notification
> system). Collectors listens to queue's topic and receives those messages.
> I'd like to:
>
>   - write plugin for agent to send proper alert when metering data hits
>   proper rules (just like in above scenario with 15% free ram)
>   - write service which will consume this alert (listen proper topic in
>   queue) and fetch data from ceilometer collector?? (or it's backend via API)
>   and find out which VM should be migrated and where (or delegate "Where" to
>   nova-scheduler)

I could help out with work on rules and receiving guest statistics if
ceilometer guys are ok with this extension.

> Another idea. Are there any Host states in Openstack? For example
> "Maintenance state"? Besides scenarios with free ram, cpu and other
> resources there is also typical example of how useful this mechanism could
> be with maintenance state.
> *Scenario:*
> 1. Administrator changes HostA state to "maintenance".
> 2. There goes alert "HostA maintenance".
> 3. Central collector or central daemon receives Alert about maintenance.
> 4. Central collector gets list of VMs on HostA.
> 5. Central collectors start migration all VMs from HostA to other Hosts (or
> invoke nova-scheduler to reassign VM from HostA to other hosts, but in this
> case we need to change or implement new filter in nova-scheduler)
> 6. Administrator changes HostA state to "active".
> 7. Openstack starts new machines on this node (this is now done via
> nova-scheduler) or migrate VMs from other overloaded hosts to HostA.
>
> Everything should be event-driven (alert-notification, host-state-change,
> administrator's action).
>
> I'm also wondering what about this
> ResourceMonitorAlertsandNotifications<http://wiki.openstack.org/ResourceMonitorAlertsandNotifications>,
> because when I'm looking at schema, it has everything what I need, but
> blueprint link is broken. Is this idea obsolete? Or is it implemented?
> (this was created 2nd April 2012, so I think it's not implemented yet).

Me too (I also have been wondering about it), I was quite keen on this
and contacted the authors, but no response...

Regards
Angus Salkeld

> First question: Does it make sense? Are such mechanisms implemented in
> Openstack right now? Or is it worth to implement?
> Second question (if first's answer isn't negative): How and when can I use
> ceilometer to do what I've described? From meeting's
> topics<http://wiki.openstack.org/Meetings/MeteringAgenda>I know that
> yesterday you were deciding what to use as notification bus,
> and in june you will be thinking about storage backend, how is it going on
> right now? Is such scenario possible in ceilometer (I mean plugins in
> agents)? Collecting data not only from guests but also from hosts.
>
> *Cheers,
> *
> *
> *

> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp