yahoo-eng-team team mailing list archive

[Heitor Alves de Siqueira <2098892@xxxxxxxxxxxxxxxxxx>]

** Changed in: nova (Ubuntu)
       Status: New => Fix Released

** Changed in: nova (Ubuntu Jammy)
   Importance: Undecided => Medium

** Changed in: nova (Ubuntu Jammy)
       Status: New => In Progress

** Changed in: nova (Ubuntu Jammy)
     Assignee: (unassigned) => Edward Hope-Morley (hopem)

** Changed in: nova (Ubuntu Noble)
   Importance: Undecided => Medium

** Changed in: nova (Ubuntu Noble)
       Status: New => In Progress

** Changed in: nova (Ubuntu Noble)
     Assignee: (unassigned) => Edward Hope-Morley (hopem)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2098892

Title:
  "TypeError: virNodeDeviceLookupByName() argument 2 must be str or
  None, not Proxy" from update_available_resource with mdevs

Status in Ubuntu Cloud Archive:
  New
Status in Ubuntu Cloud Archive caracal series:
  New
Status in Ubuntu Cloud Archive dalmatian series:
  New
Status in Ubuntu Cloud Archive yoga series:
  New
Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) 2024.1 series:
  Fix Committed
Status in OpenStack Compute (nova) 2024.2 series:
  Fix Committed
Status in OpenStack Compute (nova) antelope series:
  In Progress
Status in OpenStack Compute (nova) bobcat series:
  Fix Released
Status in nova package in Ubuntu:
  Fix Released
Status in nova source package in Jammy:
  In Progress
Status in nova source package in Noble:
  In Progress

Bug description:
  [Impact]

  The fix in bug 2091033 led to a regression that is fixed by this
  patch. In order to backport the former we also need this patch so both
  are being proposed for SRU at the same time.

  [Test Plan]

   * Deploy Openstack Yoga and ensure that "TypeError: virNodeDeviceLookupByName() argument 2 must be str or None, not Proxy" does not appear in /var/log/nova/nova-compute.log
   * The default candence for the resource tracker is 60s so need to wait at least that long for this to occur.
   * Once you see the log "Auditing locally available compute resources for" the resource tracker has started and it finishes with "Hypervisor/Node resource view:"

  [Regression Potential]

  This patch addresses a regression identified in the original version
  of the fix for bug 2091033. This reverts the change that caused the
  regression and updates the unit tests to catch it. Potential
  regressions should be spotted by regular package testing.

  --------------------------------------------------------------------------

  The patch meant to fix https://bugs.launchpad.net/nova/+bug/2091033
  unfortunately has broken the nova-compute update_available_resource
  periodic task when mdevs are involved with a traceback like shown
  below.

  We are working on a partial revert that will be backported to the same
  branches where the original patch was merged.

  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager Traceback (most recent call last):
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 10530, in _update_available_resource_for_node
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager self.rt.update_available_resource(context, nodename,
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/compute/resource_tracker.py", line 889, in update_available_resource
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager self._update_available_resource(context, resources, startup=startup)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py", line 414, in inner
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager return f(*args, **kwargs)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/compute/resource_tracker.py", line 994, in _update_available_resource
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager self._update(context, cn, startup=startup)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/compute/resource_tracker.py", line 1303, in _update
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager self._update_to_placement(context, compute_node, startup)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/retrying.py", line 49, in wrapped_f
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager return Retrying(*dargs, **dkw).call(f, *args, **kw)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/retrying.py", line 206, in call
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager return attempt.get(self._wrap_exception)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/retrying.py", line 247, in get
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager six.reraise(self.value[0], self.value[1], self.value[2])
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/six.py", line 709, in reraise
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager raise value
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/retrying.py", line 200, in call
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/compute/resource_tracker.py", line 1216, in _update_to_placement
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager self.driver.update_provider_tree(prov_tree, nodename)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 9120, in update_provider_tree
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager self._update_provider_tree_for_vgpu(
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 9529, in _update_provider_tree_for_vgpu
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager inventories_dict = self._get_gpu_inventories()
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 8199, in _get_gpu_inventories
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager count_per_dev = self._count_mdev_capable_devices(enabled_mdev_types)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 8150, in _count_mdev_capable_devices
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager mdev_capable_devices = self._get_mdev_capable_devices(
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 8392, in _get_mdev_capable_devices
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager device = self._get_mdev_capabilities_for_dev(name, types)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 8361, in _get_mdev_capabilities_for_dev
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager virtdev = self._host.device_lookup_by_name(devname)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/host.py", line 1253, in device_lookup_by_name
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager return self.get_connection().nodeDeviceLookupByName(name)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 193, in doit
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager result = proxy_call(self._autowrap, f, *args, **kwargs)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 151, in proxy_call
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager rv = execute(f, *args, **kwargs)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 132, in execute
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager six.reraise(c, e, tb)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/six.py", line 709, in reraise
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager raise value
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 86, in tworker
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager rv = meth(*args, **kwargs)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager File "/usr/lib64/python3.9/site-packages/libvirt.py", line 5201, in nodeDeviceLookupByName
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager ret = libvirtmod.virNodeDeviceLookupByName(self._o, name)
  2025-02-18 20:43:33.462 2 ERROR nova.compute.manager TypeError: virNodeDeviceLookupByName() argument 2 must be str or None, not Proxy

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2098892/+subscriptions