yahoo-eng-team team mailing list archive

[Balazs Gibizer <2115729@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

stack@aio:~$ grep device_spec /etc/nova/nova-cpu.conf 
#device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:06:", "physical_network": "physnet2"}
#device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:07:"}
#device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:08:"}
#device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:09:"}
#device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:0a:"}
#device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:0b:"}
#device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:0c:"}
#device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:0d:"}
device_spec = {}
stack@aio:~$ mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
80
stack@aio:~$ grep device_spec /etc/nova/nova-cpu.conf 
#device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:06:", "physical_network": "physnet2"}
#device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:07:"}
#device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:08:"}
#device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:09:"}
#device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:0a:"}
#device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:0b:"}
#device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:0c:"}
#device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:0d:"}
#device_spec = {}
stack@aio:~$ sudo systemctl restart devstack@n-cpu
stack@aio:~$ mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
58
stack@aio:~$ date && mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
Tue Jul  1 12:42:13 UTC 2025
40
stack@aio:~$ date && mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
Tue Jul  1 12:42:31 UTC 2025
40
stack@aio:~$ date && mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
Tue Jul  1 12:43:09 UTC 2025
20
stack@aio:~$ date && mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
Tue Jul  1 12:50:35 UTC 2025
0

It feels like nova-compute at each periodic run deletes half of the
removed deviced only. But eventually removes them all.

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: pci

** Tags added: pci

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2115729

Title:
  Removing a lot of PCI devices via changing device_spec seems to be
  slow

Status in OpenStack Compute (nova):
  New

Bug description:
  stack@aio:~$ grep device_spec /etc/nova/nova-cpu.conf 
  #device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:06:", "physical_network": "physnet2"}
  #device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:07:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:08:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:09:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:0a:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:0b:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:0c:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:0d:"}
  device_spec = {}
  stack@aio:~$ mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
  80
  stack@aio:~$ grep device_spec /etc/nova/nova-cpu.conf 
  #device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:06:", "physical_network": "physnet2"}
  #device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:07:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:08:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:09:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:0a:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10ca", "address": "0000:0b:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:0c:"}
  #device_spec = { "vendor_id":"8086", "product_id":"10c9", "address": "0000:0d:"}
  #device_spec = {}
  stack@aio:~$ sudo systemctl restart devstack@n-cpu
  stack@aio:~$ mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
  58
  stack@aio:~$ date && mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
  Tue Jul  1 12:42:13 UTC 2025
  40
  stack@aio:~$ date && mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
  Tue Jul  1 12:42:31 UTC 2025
  40
  stack@aio:~$ date && mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
  Tue Jul  1 12:43:09 UTC 2025
  20
  stack@aio:~$ date && mysql -e 'select deleted, address from nova_cell1.pci_devices where compute_node_id = 1 and deleted = 0;' | wc -l
  Tue Jul  1 12:50:35 UTC 2025
  0

  It feels like nova-compute at each periodic run deletes half of the
  removed deviced only. But eventually removes them all.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2115729/+subscriptions