← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1620692] [NEW] Debug logging in scheduler kills performance

 

Public bug reported:

When using the filter_scheduler with 400 hosts (not a very large number)
and debug logging turned on, scheduling times start taking a very long
time. With debug logging on, select_destinations() can swing anywhere
between 3 and 18 seconds. With debug logging off, select_destinations()
takes 0-4 seconds (http://paste.openstack.org/show/566153/).

The main problem is
https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L162-L178
because in a 400 host environment, it's trying to log debug 1600 times
on every instance boot.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1620692

Title:
  Debug logging in scheduler kills performance

Status in OpenStack Compute (nova):
  New

Bug description:
  When using the filter_scheduler with 400 hosts (not a very large
  number) and debug logging turned on, scheduling times start taking a
  very long time. With debug logging on, select_destinations() can swing
  anywhere between 3 and 18 seconds. With debug logging off,
  select_destinations() takes 0-4 seconds
  (http://paste.openstack.org/show/566153/).

  The main problem is
  https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L162-L178
  because in a 400 host environment, it's trying to log debug 1600 times
  on every instance boot.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1620692/+subscriptions


Follow ups