openstack team mailing list archive

[Sandy Walsh <sandy.walsh@xxxxxxxxxxxxx>]

(opening this thread up to the mailing list as it's a good conversation)

Yes, if you look at HostFilter there is a helper method in all drivers to map from InstanceType to whatever type of query format the driver expects. The weights will need a standardized format I suspect. 

I'm open to suggestions here, but I thought the specs (aka host query) would be an optional argument on the POST /server command (boot) ... if present it would do the more advanced scheduler. Otherwise, it would fall back to the most-common-denominator InstanceType format: ram, disk, bandwidth. 

Specs are qualifications of capabilities (ram > 5G, disk > 500G, etc)

We're hoping to keep capabilities as Key Value pairs since so many users want so many different things. If there is agreement we could move some capabilities into the InstanceType table. 

Thoughts?
-S

PS> Side note: we really need to clean up the parameter lists coming from nova.compute.api:create() into the schedulers ... it's *args/**kwargs hell right now. 

________________________________________
From: Lorin Hochstein [lorin@xxxxxxx]
Sent: Friday, May 13, 2011 11:50 AM
To: Sandy Walsh
Cc: dodcs@xxxxxxxxxxxxxxxxxxxx; Rick Harris
Subject: Re: dist-scheduler-1

Sandy:

Cool...

On the spec* vs. InstanceType topic, I assume that somewhere along the line, the code will need to do a mapping from instance type to spec. Where are you thinking of encoding the information about  capabilities required for each instance type? For example, are you planning on adding something like an InstanceTypeSpecs table that maps types to the capabilities required?

* Is "spec" short for "specification"? I assume from code context that it refers to the capabilities that are needed to satisfy a request to run an instance, but wasn't 100% sure.

Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin

On May 13, 2011, at 9:43 AM, Sandy Walsh wrote:

> I've fixed those things in my branch. Rick will pull before merge-prop'ing.
>
> Thanks again!
> -S
>
>
> ________________________________________
> From: Sandy Walsh
> Sent: Friday, May 13, 2011 9:33 AM
> To: Lorin Hochstein
> Cc: dodcs@xxxxxxxxxxxxxxxxxxxx; Rick Harris
> Subject: RE: dist-scheduler-1
>
> Nice! All correct Lorin.
>
> Rick just added some more tests. We'll fix that up before issuing the merge prop today.
>
> The whole spec vs. InstanceType thing is something we're still working through. We don't want to bust existing stuff but still need to add the, more flexible, query mechanism. That'll all emerge over the next couple of branches.
>
> Our next step will be getting HostFilter and Rick's CostScheduler tied in (via two parallel merge props)
>
> Great to have your input on this!
>
> -S
> ________________________________________
> From: Lorin Hochstein [lorin@xxxxxxx]
> Sent: Friday, May 13, 2011 1:12 AM
> To: Sandy Walsh
> Cc: dodcs@xxxxxxxxxxxxxxxxxxxx
> Subject: dist-scheduler-1
>
> Sandy:
>
> I've been playing with your dist-scheduler-1 branch, to see if I can use what's there to do a quick proof-of-concept of heterogeneous nodes using it. (I'm starting off by seeing if I can just get a dead-simple zone scheduler working).
>
>  I realize it's still in active development, but I wanted to let you know about the following issues I saw in the code that didn't have TODO associated:
>
> 1.In zone_aware_scheduler.py:
>
> ZoneAwareScheduler.schedule_run_instance takes specs=None as the default, but this will barf when it hits  the "if 'blob' in specs" in the method body. Maybe specs={} instead?
>
> 2. ZoneAwareScheduler.weigh_hosts has the following docstring:
>
>        """Derived classes must override this method and return
>           a lists of hosts in [(weight, hostname)] format."""
>
> However, based on other parts of the code, it seems thatthis function should return a list of dicts [{'weight': w, 'hostname':h}] instead of a list of pairs.
>
> I created a branch at lp:~lorinh/nova/dist-sched-1 that I'm hacking away at.
>
> Lorin
> --
> Lorin Hochstein, Computer Scientist
> USC Information Sciences Institute
> 703.812.3710
> http://www.east.isi.edu/~lorin
>



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse@xxxxxxxxxxxxx, and delete the original message.
Your cooperation is appreciated.