openstack team mailing list archive

[Chris Behrens <cbehrens@xxxxxxxxxxxx>]

On Feb 25, 2013, at 6:39 PM, Joe Gordon <jogo@xxxxxxxxxxxxxxxx> wrote:

> 
> It looks like the scheduler issues are related to the rabbitmq issues.   "host 'qh2-rcc77' ... is disabled or has not been heard from in a while"
> 
> What does 'nova host-list' say?   the clocks must all be synced up?

Good things to check.  It feels like something is spinning way too much within this filter, though.  This can also cause the above message.  The scheduler pulls all of the records before it starts filtering… and if there's a huge delay somewhere, it can start seeing a bunch of hosts as disabled.

The filter doesn't look like a problem.. unless there's a large amount of aggregate metadata… and/or a large amount of key/values for the instance_type's extra specs.   There *is* a DB call in the filter.  If that's blocking for an extended period of time, the whole process is blocked…  But I suspect by the '100% cpu' comment, that this is not the case…  So the only thing I can think of is that it returns a tremendous amount of metadata.

Adding some extra logging in the filter could be useful.

- Chris