yahoo-eng-team team mailing list archive

[Michael Quiniola <2023414@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Openstack Focal/Ussuri
Libvirt

When a device (Network or Disk) is attached to a running instance and
then the instance is shutoff (via the OS or Nova), the re-render of the
xml file reorders the devices. Ubuntu/Linux has the ability to match the
network interface to the correct device (when configured properly) but
Windows does not. Upon shutdown and start of these instances the
instance follows the order of enumeration of the device and the OS then
attaches the wrong network configuration to (what it thinks) is the
correct interface.

Steps to reproduce:
1) Start an instance
2) Add another Network Interface to that instance while it is running.
3) Shutdown the instance
4) Start the instance again and observe the devices in the instance.

On Windows machines this immediately causes network connection issues as
the wrong configuration is being used on the wrong device.

We have not tested this with Nova/VMWare.

Per @krenshaw:

"The PCI slots are being reordered when Nova rebuilds the VM after any
sort of hard stop (openstack server stop, evacuate, etc). This causes
both the MAC interchange and disk offline issues.

The reason this occurs is that Nova redefines the VM after stop events,
up to and including a hard reboot[0]. When this occurs, the VM is
regenerated with all currently attached devices, making them sequential
within the device type.

This causes reordering when an instance has had volumes and/or networks
attached and detached, as devices that are attached after boot are added
at the end of the list of PCI slots. On rebuild, these move to PCI slots
in sequential order, regardless of the attach/detach order.

Having checked the Nova code, Nova doesn't store PCI information for
"regular" non-PCI-passthrough devices. This includes NICs and volumes.
Adding this capability would be a feature request with no guarantee of
implementation."


We (@setuid @krenshaw) believe it is the metadata that nova is passing to libvirt to re-render the XML file.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2023414

Title:
  Devices attached to running instances get reordered

Status in OpenStack Compute (nova):
  New

Bug description:
  Openstack Focal/Ussuri
  Libvirt

  When a device (Network or Disk) is attached to a running instance and
  then the instance is shutoff (via the OS or Nova), the re-render of
  the xml file reorders the devices. Ubuntu/Linux has the ability to
  match the network interface to the correct device (when configured
  properly) but Windows does not. Upon shutdown and start of these
  instances the instance follows the order of enumeration of the device
  and the OS then attaches the wrong network configuration to (what it
  thinks) is the correct interface.

  Steps to reproduce:
  1) Start an instance
  2) Add another Network Interface to that instance while it is running.
  3) Shutdown the instance
  4) Start the instance again and observe the devices in the instance.

  On Windows machines this immediately causes network connection issues
  as the wrong configuration is being used on the wrong device.

  We have not tested this with Nova/VMWare.

  Per @krenshaw:

  "The PCI slots are being reordered when Nova rebuilds the VM after any
  sort of hard stop (openstack server stop, evacuate, etc). This causes
  both the MAC interchange and disk offline issues.

  The reason this occurs is that Nova redefines the VM after stop
  events, up to and including a hard reboot[0]. When this occurs, the VM
  is regenerated with all currently attached devices, making them
  sequential within the device type.

  This causes reordering when an instance has had volumes and/or
  networks attached and detached, as devices that are attached after
  boot are added at the end of the list of PCI slots. On rebuild, these
  move to PCI slots in sequential order, regardless of the attach/detach
  order.

  Having checked the Nova code, Nova doesn't store PCI information for
  "regular" non-PCI-passthrough devices. This includes NICs and volumes.
  Adding this capability would be a feature request with no guarantee of
  implementation."

  
  We (@setuid @krenshaw) believe it is the metadata that nova is passing to libvirt to re-render the XML file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2023414/+subscriptions