yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #21685
[Bug 1161664] Re: Rescheduling can DDOS itself
This isn't really a bug. If you have a reproduce with an expected
behavior and actually behavior maybe we can turn it into a bug.
** Changed in: nova
Status: Triaged => Opinion
** Changed in: nova
Importance: Low => Wishlist
** Tags added: scheduler
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1161664
Title:
Rescheduling can DDOS itself
Status in OpenStack Compute (Nova):
Opinion
Bug description:
Due to the way nova currently handles rescheduling there is a
tendency, when a large number of compute nodes need to reschedule (for
whatever error) that they then swamp the message queue (and nova
scheduler) with rescheduling messages. This can cascade to where
further rescheduling messages will occur (and repeat...) or until the
MQ piles up and/or the scheduler falls over.
Even with a reschedule 'count' under situations when rescheduling is
happening on mass the rescheduling itself can cause more problems for
your system than it helps solve (aka, just leave the scheduled
instance in error state). Likely the way to do this in a more
centralized manner (aka with a orchestration unit that can do this
rescheduling on behalf of the request) can help rate limit itself and
its requests to the scheduler for new locations to schedule to. Having
each compute node perform this same operation means rate limiting is
not possible (and allows for your own system to DDOS itself).
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1161664/+subscriptions