yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

This goes back to Rocky when the CPUWeigher was added:

https://review.opendev.org/#/c/379525/

** Changed in: nova
       Status: New => Triaged

** Changed in: nova
   Importance: Undecided => High

** Also affects: nova/rocky
   Importance: Undecided
       Status: New

** Also affects: nova/stein
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1834694

Title:
  TypeError: unsupported operand type(s) for *: 'int' and 'NoneType' in
  CPUWeigher

Status in OpenStack Compute (nova):
  Triaged
Status in OpenStack Compute (nova) rocky series:
  New
Status in OpenStack Compute (nova) stein series:
  New

Bug description:
  This is similar to bug 1834691 in that the CPUWeigher can hit this
  error:

  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server [None req-df4ac006-fddf-4e6f-9c5e-1c352be60d9b admin admin] Exception during message handling
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 165, in _p
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 274, i
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 194, i
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 228, in in
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/scheduler/manager.py", line 192, in select_destinations
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     allocation_request_version, return_alternates)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 96, in select_destinations
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     allocation_request_version, return_alternates)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 210, in _schedule
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     hosts = self._get_sorted_hosts(spec_obj, hosts, num)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 449, in _get_sorted_hosts
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     spec_obj)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/scheduler/host_manager.py", line 601, in get_weighed_hosts
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     hosts, spec_obj)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/weights.py", line 137, in get_weighed_objects
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     weights = weigher.weigh_objects(weighed_objs, weighing_properties)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/weights.py", line 107, in weigh_objects
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     weight = self._weigh_object(obj.obj, weight_properties)
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/scheduler/weights/cpu.py", line 44, in _weigh_object
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server     host_state.vcpus_total * host_state.cpu_allocation_ratio -
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'
  Jun 28 20:26:07 devstack nova-scheduler[24099]: ERROR oslo_messaging.rpc.server 

  
  If the HostState.cpu_allocation_ratio is not set here:

  https://github.com/openstack/nova/blob/231908a7f42b8329a58e62c8a89204fdeaa16dca/nova/scheduler/host_manager.py#L256

  Because we return early here:

  https://github.com/openstack/nova/blob/231908a7f42b8329a58e62c8a89204fdeaa16dca/nova/scheduler/host_manager.py#L193

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1834694/+subscriptions