openstack team mailing list archive

[Jan Drake <jan_drake@xxxxxxxxxxx>]

Stated...

1: Thanks to everyone for contextual examples

Proposed...

1: Additional capability descriptors should always be at a group level... a group can exist of one but most use cases will have more than one
2: API should be created to access this info allowing implementation to come from anywhere (config file, chef, rpath, metadata from hypervisors, etc)
3: Default implementation should pull data from well known and uniform location... which is? (do chime in)

I totally grok the need for host decorators but also need the source of the decorators to be configurable based on my specific environment around openstack.

JanMan

...after rockin' a night with Devin.  Any fuzziness of communication is merely a temporary side effect.  If required, just squint a bit and I should make better sense.



On Apr 3, 2012, at 6:45 AM, "Day, Phil" <philip.day@xxxxxx> wrote:

> Hi John,
>  
> Maybe the problem with host aggregates is that it too quickly became something that was linked to hypervisor capability, rather than being the more general mechanism of which one form of aggregate could be linked to hypervisor capabilities ?
>  
> Should we have a “host aggregates 2.0” session at the Design Summit ?
>  
> Phil
>  
>  
> From: John Garbutt [mailto:John.Garbutt@xxxxxxxxxx] 
> Sent: 03 April 2012 11:43
> To: Day, Phil; Jan Drake; Lorin Hochstein
> Cc: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Openstack] Limit flavors to specific hosts
>  
> +1 to REST API and nova cli support
>  
> +1 to exposing Glance metadata to the scheduler. The main reason I thought about this was a multi-hypervisor cloud (matching images to the correct hypervisor). But some images may require more than 1GB RAM, etc.
>  
> I see an aggregate is a groups of compute hosts within a single availability zone (assumed to have similar capabilities). With XenServer, it does build up a resource pool as you add hosts into the aggregate (you could add a flag to stop that happening, if that isn’t required in your cloud).
>  
> I always expected to look at making the scheduler aware of the host-aggregate metadata. That would allow different groups of hosts, across multiple availability zones, to have the same extra capabilities. You could add a similar metadata service on individual hosts. Then you could look at cascading aggregate metadata and adding using the host metadata as an override to create an overall set of metadata for the scheduler, but that might be gilding the lily a bit…
>  
> Also, maybe flavor is not expressive enough for the billing and scheduling, maybe we need some concept like “server requirements”.
>  
> It might be something like:
> ·         User requests an image and flavor, and perhaps particular networking configuration
> ·         Glance meta for the image is collected
> o   Could tell you that a flavor is not valid for this image
> o   Could tell you extra requirements, like: requires XenServer, or requires KVM
> ·         This builds up some “service requirements” for this instance
> o   Union of info from users request and image meta-data
> o   These “server requirements” can be used for the billing, instead of just flavor id
> ·         Host metadata collected
> o   use capabilities (including those added in config file)
> o   aggregate metadata
> o   disabled flag,  (+ host metadata overrides?)
> ·         Scheduler then filters out hosts that can’t satisfy the service requirements
>  
> Good points:
> ·         a clear catalog of service offerings to choose from
> ·         stops the number of different flavors ballooning (with/without GPU, XenServer or KVM, 1GB or 2GB, etc)
> Bad points:
> ·         billing is more complicated
> o   no longer get flavors => price
> ·         seems too complex
> o   probably means there are some useful simplifications we could make
> o   could just use flavor, but you get a ballooning of the number of flavors (GPU vs no GPU, XenServer vs KVM, 1GB vs 2GB)
>  
> John
>  
> From: openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Day, Phil
> Sent: 03 April 2012 10:33
> To: Jan Drake; Lorin Hochstein
> Cc: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Openstack] Limit flavors to specific hosts
>  
> Yes - Its more generic that hypervisor capabilities –  my main problem with Host Aggregates is that it limits it to some specific 1:1 groupings based on hypervisor functionality.
>  
> Use cases I want to be able to cover include:
>  
> -          Rolling new hardware through an existing cluster, and limiting some new flavors (which might for example provide higher network bandwidth) to just those servers
>  
> -          Providing a range of flavours that are dependent on specific hardware features (GPU)
>  
> -          There may be a further group that couples flavour and/or images to host groups – for example it’s possible to imagine a scenario where an image is only licensed to some specific subset of servers, or where a subset of nodes are running LXC (in which case the image is in effect pre-defined).     In this case the image metadata could, for example, specify the flavors that it can be used with, and those flavors are in turn limited to specific hosts.   I don’t really like this model of linking Glance objects (images) to Nova Objects (flavors), but I’m not sure what an alternative would be.
>  
> On the config file vs REST API for configuration debate (maybe this needs to be a Design Summit subject in its own right), I agree that we should make a distinction between items that are deploy time configuration (which hypervisor to use, network driver, etc) and items that could change whilst the system is running (rate limits is a good example).     I don’t however see this as being an REST API vs config file issue  - more a configuration repository issue.   I’d also add that anything which is going to be configured via a REST API needs to also provide a command line tool to drive that interface – so that out of the box the system can be installed and configured via the tools and scripts shipped with it.
>  
> Phil
>  
>  
>  
> From: openstack-bounces+philip.day=hp.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+philip.day=hp.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Jan Drake
> Sent: 03 April 2012 02:23
> To: Lorin Hochstein
> Cc: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Openstack] Limit flavors to specific hosts
>  
> If I understand this correctly, the motivation is to be able to provide a hint to schedulers on host-level appropriateness based on information external to that found in the hyperviser.  
>  
> Right/Wrong/Close?
>  
> It would help to have a real-world example of where basic host resource  evalution for scheduling would cause a situation requiring the host-level hard-coding of what is essentially a flavor-constraint.
>  
> I'll hold further thoughts for downstream.
>  
>  
> Jan
> 
> 
> On Apr 2, 2012, at 6:06 PM, Lorin Hochstein <lorin@xxxxxxxxxxxxxxxxxx> wrote:
> 
> Just created a blueprint for this:
>  
> https://blueprints.launchpad.net/nova/+spec/host-capabilities-api
>  
>  
> Take care,
>  
> Lorin
> --
> Lorin Hochstein
> Lead Architect - Cloud Services
> Nimbis Services, Inc.
> www.nimbisservices.com
>  
>  
>  
> 
>  
> On Apr 2, 2012, at 3:29 PM, Jay Pipes wrote:
>  
> 
> Can I add a feature request to the below thoughtstream? Can we make it so that the management of these things can be done outside of config files? i.e. via a REST API with some simple middleware exposing the particular scheduler nodes' understanding of which capabilities/filters it is using to apply its scheduling algorithm?
> 
> Making changes to configuration files works OK for simple uses and testing, not so much for on-demand operations :) I say this after grumbling about similar configuration obstacles with ratelimits.
> 
> Best,
> -jay
> 
> On 04/02/2012 02:37 PM, Chris Behrens wrote:
> 
> I have some plans for being able to set arbitrary "capabilities" for
> hosts via nova.conf that you can use to build scheduler filters.
>  
> Right now, there are capabilities, but I believe we're only creating
> these from hypervisor stats. You can filter on those today. What I'm
> planning on adding is a way to specify additional keyname/value pairs in
> nova.conf to supplement the capabilities we build from hypervisor stats.
> You could set things like this in your nova.conf:
>  
> --host_capabilities=instance_type_ids=1,2,3;keyX;keyY=something
>  
> etc. Since capabilities are already passed to scheduler rules, you could
> add some basic filters that do:
>  
> if 'instance_type_ids' in capabilities and instance_type.id not in
> capabilities['instance_type_ids']:
> return False
>  
> Since host_capabilities are just arbitrary keyname/value pairs, you can
> pretty much add anything you want to --host_capabilities and then write
> some matching scheduler filter rules.
>  
> That's the basic idea, anyway. The exact same behavior will apply to
> 'cells' and the cells scheduler as well. (Except you'll have
> cells_capabilities= somewhere (prob nova.conf for the cells service).
>  
> - Chris
>  
>  
> On Apr 2, 2012, at 10:36 AM, Day, Phil wrote:
>  
> Hi Folks,
> I’m looking for a capability to limit some flavours to some hosts. I
> want the mapping to be as flexible as possible, and work within a
> zone/cell (I don’t want to add zones just to get this mapping). For
> example I want to express something like:
> Host_1 supports flavours A, C
> Host_2 supports flavours A, B
> Host_3 supports flavours A, B, C
> Host_4 supports flavours D
> Ideally there would be some form of grouping to sets of flavours:
> Flavour_A is part of Flavour_Sets 1, 2, 3
> Flavour_B is part of Flavour_Sets 2, 3
> Flavour_C is part of Flavour_Sets 1, 3, 4
> Host_1 supports flavour Set 1
> Host_2 supports flavour Set 2
> Host_3 supports flavour Set 3
> Host_4 supports flavour Set 4
> >From the Essex design summit I thought that host aggregates was going to give this sort of capability, but having looked through the code that seems to be quite tightly coupled with specific hypervisor functionality, whereas this is purely a scheduler feature.
> I can see that I could define flavour group membership through the
> instanace_type_extra_specs, but not how to then associate these with
> specific hosts.
> I know I’m a tad behind some of the recent changes – so before
> suggesting a design summit session on this I thought I’d ask - is
> there something that already does this type of mapping ?
> Cheers,
> Phil
> _______________________________________________
> Mailing list:https://launchpad.net/~openstack
> Post to :openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe :https://launchpad.net/~openstack
> More help :https://help.launchpad.net/ListHelp
>  
>  
>  
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>  
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp