yahoo-eng-team team mailing list archive

[sean mooney <smooney@xxxxxxxxxx>]

Public bug reported:

The q35 machine type support native pcie instead of legacy ahci
based pci hotplug. This has several advantages and one majour disadvantage
with the new pcie approch you need to pre allocate the pcie slot so that
they will be available for use with hotplug if needed.

to support this a new num_pcie_ports config option was added to the libvirt section of
the nova.conf

https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.num_pcie_ports

the default value of this config is 0 which mean we use libvirts default.
libvirts default is to allocate 1 free pcie port as a result by default
you cannot attach more then 1 device without hard rebooting the vm.

previously when using the pc machine type with the i440fx chipset it was possible to attach
multiple interfaces or volumes. as a result the end user behavior has changed
as observed by the failrure in tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces_by_network_port with the default setting and q35 enabled as 
reported in this downstream bug https://bugzilla.redhat.com/show_bug.cgi?id=1716356

to fix this the suggestion is to set the max value and default value of the
num_pcie_ports config option to 32

based on some minimal local testing the memory usage of this change is ~0.4MB per port or ~12.5 mb per vm in addtion qemu overhead. this is based on testing done with libvirt directly with memroy preallocation enabeld
for a 2G guest with the pc machinetype and i440fx chipset  total memroy of 2036 MB was observed, 
q35 4 ports (the default value that will be calulated by libvirt for the default devices)  increesed this to  2056 MB and q35 32 ports to 2066 MB

as such this i a minimal overhead increase which can still be controlled
by setting the config to a lower value explicitly.

** Affects: nova
     Importance: Low
     Assignee: Kashyap Chamarthy (kashyapc)
         Status: In Progress


** Tags: libvirt

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1831886

Title:
  default pcie hotplug behavior changes when using q35

Status in OpenStack Compute (nova):
  In Progress

Bug description:
  The q35 machine type support native pcie instead of legacy ahci
  based pci hotplug. This has several advantages and one majour disadvantage
  with the new pcie approch you need to pre allocate the pcie slot so that
  they will be available for use with hotplug if needed.

  to support this a new num_pcie_ports config option was added to the libvirt section of
  the nova.conf

  https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.num_pcie_ports

  the default value of this config is 0 which mean we use libvirts default.
  libvirts default is to allocate 1 free pcie port as a result by default
  you cannot attach more then 1 device without hard rebooting the vm.

  previously when using the pc machine type with the i440fx chipset it was possible to attach
  multiple interfaces or volumes. as a result the end user behavior has changed
  as observed by the failrure in tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces_by_network_port with the default setting and q35 enabled as 
  reported in this downstream bug https://bugzilla.redhat.com/show_bug.cgi?id=1716356

  to fix this the suggestion is to set the max value and default value of the
  num_pcie_ports config option to 32

  based on some minimal local testing the memory usage of this change is ~0.4MB per port or ~12.5 mb per vm in addtion qemu overhead. this is based on testing done with libvirt directly with memroy preallocation enabeld
  for a 2G guest with the pc machinetype and i440fx chipset  total memroy of 2036 MB was observed, 
  q35 4 ports (the default value that will be calulated by libvirt for the default devices)  increesed this to  2056 MB and q35 32 ports to 2066 MB

  as such this i a minimal overhead increase which can still be
  controlled by setting the config to a lower value explicitly.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1831886/+subscriptions