← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #94382

[Bug 2075504] [NEW] Unhselve to specific host can fail with oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

When a libvirt based VM is unshelved to a ironic compute host then nova
does not fail the operation gracefully with NoValidHost, but instead the
scheduler fails with a stack trace:

2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server [None req-a45169a4-4024-48d0-bd04-a081d12bfbbc 99a273a7d2014e82a7d043c6e93a6ff6 be4c3131a4cd4f9f9652d3f6f4987f42 - - default default] Exception during message handling: oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 244, in inner
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 224, in select_destinations
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     selections = self._select_destinations(
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 251, in _select_destinations
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     selections = self._schedule(
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 388, in _schedule
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     hosts = self._get_sorted_hosts(spec_obj, hosts, num)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 672, in _get_sorted_hosts
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     filtered_hosts = self.host_manager.get_filtered_hosts(host_states,
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/host_manager.py", line 590, in get_filtered_hosts
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     hosts = _get_hosts_matching_request(hosts, requested_node)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/host_manager.py", line 562, in _get_hosts_matching_request
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     requested_destination.node)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 67, in getter
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     self.obj_load_attr(name)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/objects/request_spec.py", line 1140, in obj_load_attr
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     self.obj_set_defaults(attrname)
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 588, in obj_set_defaults
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     raise exception.ObjectActionError(
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server [00m


The unshelve code creates a Destination object with only the host field set and does not set the node filed. 
https://github.com/openstack/nova/blob/db263370ec9e306eb03addf34b197890996054d7/nova/compute/api.py#L4660-L4662

However the same code seems to work without the stack trace if the
target host is a libvirt compute.

Seen on 2023.1 in tempest test test_unshelve_to_specific_host

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2075504

Title:
  Unhselve to specific host can fail with
  oslo_versionedobjects.exception.ObjectActionError: Object action
  set_defaults failed because: No default set for field node

Status in OpenStack Compute (nova):
  New

Bug description:
  When a libvirt based VM is unshelved to a ironic compute host then
  nova does not fail the operation gracefully with NoValidHost, but
  instead the scheduler fails with a stack trace:

  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server [None req-a45169a4-4024-48d0-bd04-a081d12bfbbc 99a273a7d2014e82a7d043c6e93a6ff6 be4c3131a4cd4f9f9652d3f6f4987f42 - - default default] Exception during message handling: oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 244, in inner
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 224, in select_destinations
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     selections = self._select_destinations(
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 251, in _select_destinations
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     selections = self._schedule(
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 388, in _schedule
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     hosts = self._get_sorted_hosts(spec_obj, hosts, num)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 672, in _get_sorted_hosts
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     filtered_hosts = self.host_manager.get_filtered_hosts(host_states,
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/host_manager.py", line 590, in get_filtered_hosts
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     hosts = _get_hosts_matching_request(hosts, requested_node)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/scheduler/host_manager.py", line 562, in _get_hosts_matching_request
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     requested_destination.node)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 67, in getter
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     self.obj_load_attr(name)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/nova/objects/request_spec.py", line 1140, in obj_load_attr
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     self.obj_set_defaults(attrname)
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 588, in obj_set_defaults
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server     raise exception.ObjectActionError(
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node
  2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server [00m

  
  The unshelve code creates a Destination object with only the host field set and does not set the node filed. 
  https://github.com/openstack/nova/blob/db263370ec9e306eb03addf34b197890996054d7/nova/compute/api.py#L4660-L4662

  However the same code seems to work without the stack trace if the
  target host is a libvirt compute.

  Seen on 2023.1 in tempest test test_unshelve_to_specific_host

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2075504/+subscriptions
  Thread Previous • Date Previous • • Thread Next