yahoo-eng-team team mailing list archive

[Pierre Riteau <2048097@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Description
===========

VIF unplug operations are failing on Rocky Linux 9.3 with stable/yoga.
This is caused by incompatibility between pyroute2 0.6.6 (yoga u-c) and
the Rocky Linux 9.3 kernel.

Steps to reproduce
==================

* Launch a virtual machine
* Try to resize the virtual machine


Expected result
===============

The resize operation succeeds (assuming enough resources are available).

Actual result
=============

The resize operation will fail and a similar traceback will be logged in
nova-compute:

Traceback (most recent call last):
  File "/var/lib/kolla/venv/lib/python3.9/site-packages/os_vif/__init__.py", line 110, in unplug
    plugin.unplug(vif, instance_info)
  File "/var/lib/kolla/venv/lib/python3.9/site-packages/vif_plug_ovs/ovs.py", line 445, in unplug
    self._unplug_bridge(vif, instance_info)
  File "/var/lib/kolla/venv/lib/python3.9/site-packages/vif_plug_ovs/ovs.py", line 376, in _unplug_bridge
    linux_net.delete_bridge(vif.bridge_name, v1_name)
  File "/var/lib/kolla/venv/lib/python3.9/site-packages/oslo_privsep/priv_context.py", line 271, in _wrap
    return self.channel.remote_call(name, args, kwargs,
  File "/var/lib/kolla/venv/lib/python3.9/site-packages/oslo_privsep/daemon.py", line 215, in remote_call
    raise exc_type(*result[2])
pr2modules.netlink.exceptions.NetlinkError: (95, 'Operation not supported')

Note that this was resolved in Ubuntu by backporting a fix from pyroute2
0.6.10. For more details see
https://bugs.launchpad.net/ubuntu/+source/pyroute2/+bug/1995469

We successfully reproduced the fix by backporting the following commit:
https://github.com/svinota/pyroute2/commit/1eb08312de30a083bcfddfaa9c1d5e124b6368df

Environment
===========

This should be reproducible with Nova stable/yoga at the time of writing, when running on Rocky Linux 9.3:
5.14.0-362.8.1.el9_3.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov 8 17:36:32 UTC 2023

Hypervisor is KVM:
qemu-kvm-7.2.0-14.el9_2.5.x86_64
libvirt-daemon-9.0.0-10.3.el9_2.x86_64

Storage is Ceph (Quincy).

Networking is Neutron with OpenVSwitch.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2048097

Title:
  VIF unplug operation failing on Rocky Linux 9.3 (Yoga)

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========

  VIF unplug operations are failing on Rocky Linux 9.3 with stable/yoga.
  This is caused by incompatibility between pyroute2 0.6.6 (yoga u-c)
  and the Rocky Linux 9.3 kernel.

  Steps to reproduce
  ==================

  * Launch a virtual machine
  * Try to resize the virtual machine

  
  Expected result
  ===============

  The resize operation succeeds (assuming enough resources are
  available).

  Actual result
  =============

  The resize operation will fail and a similar traceback will be logged
  in nova-compute:

  Traceback (most recent call last):
    File "/var/lib/kolla/venv/lib/python3.9/site-packages/os_vif/__init__.py", line 110, in unplug
      plugin.unplug(vif, instance_info)
    File "/var/lib/kolla/venv/lib/python3.9/site-packages/vif_plug_ovs/ovs.py", line 445, in unplug
      self._unplug_bridge(vif, instance_info)
    File "/var/lib/kolla/venv/lib/python3.9/site-packages/vif_plug_ovs/ovs.py", line 376, in _unplug_bridge
      linux_net.delete_bridge(vif.bridge_name, v1_name)
    File "/var/lib/kolla/venv/lib/python3.9/site-packages/oslo_privsep/priv_context.py", line 271, in _wrap
      return self.channel.remote_call(name, args, kwargs,
    File "/var/lib/kolla/venv/lib/python3.9/site-packages/oslo_privsep/daemon.py", line 215, in remote_call
      raise exc_type(*result[2])
  pr2modules.netlink.exceptions.NetlinkError: (95, 'Operation not supported')

  Note that this was resolved in Ubuntu by backporting a fix from
  pyroute2 0.6.10. For more details see
  https://bugs.launchpad.net/ubuntu/+source/pyroute2/+bug/1995469

  We successfully reproduced the fix by backporting the following
  commit:
  https://github.com/svinota/pyroute2/commit/1eb08312de30a083bcfddfaa9c1d5e124b6368df

  Environment
  ===========

  This should be reproducible with Nova stable/yoga at the time of writing, when running on Rocky Linux 9.3:
  5.14.0-362.8.1.el9_3.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov 8 17:36:32 UTC 2023

  Hypervisor is KVM:
  qemu-kvm-7.2.0-14.el9_2.5.x86_64
  libvirt-daemon-9.0.0-10.3.el9_2.x86_64

  Storage is Ceph (Quincy).

  Networking is Neutron with OpenVSwitch.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2048097/+subscriptions