← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 2105896] [NEW] Multiple security groups in a project cause VM build failures

 

Public bug reported:


A bug was introduced by https://review.opendev.org/c/openstack/nova/+/811521, in which duplicate security group names from a project [1] can cause VM build errors such as the one below [2] [3].

This occurs when creating a vm with a different security (by name) or by
using a uuid. In this example I used the uuid for the management sec-
group, but the vm build failed due to two security groups being the same
name in the project.


[1]
# openstack security group list --project admin
+--------------------------------------+---------------------+---------------------------------+----------------------------------+------+
| ID                                   | Name                | Description                     | Project                          | Tags |
+--------------------------------------+---------------------+---------------------------------+----------------------------------+------+
| 123139a5-1ff2-4818-bc73-7acf839ef49b | ceph-security-group | ceph-security-group             | 01115f1ff5974b5286707ef8e625f86b | []   |
| 28e42ade-5ac2-412e-80c2-74139f4d04b5 | ceph-security-group | Security group for Ceph cluster | 01115f1ff5974b5286707ef8e625f86b | []   |
| 52457b0d-9fd4-4b74-9fec-a17fdca5c86d | default             | Default security group          | 01115f1ff5974b5286707ef8e625f86b | []   |
| 9507768e-1740-4233-8840-28d58cad87cc | management          | management                      | 01115f1ff5974b5286707ef8e625f86b | []   |
+--------------------------------------+---------------------+---------------------------------+----------------------------------+------+


[2]

## openstack server show 3935c340-7521-4082-b140-8d453e792cb8 -c fault -f json
compute version 2.89 is not in supported versions: 2, 2.1
{
  "fault": {
    "code": 500,
    "created": "2025-04-01T18:25:54Z",
    "message": "Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3935c340-7521-4082-b140-8d453e792cb8.",
    "details": "Traceback (most recent call last):\n  File \"/var/lib/kolla/venv/lib/python3.12/site-packages/nova/conductor/manager.py\", line 705, in build_instances\n    raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3935c340-7521-4082-b140-8d453e792cb8.\n"
  }
}


[3]
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [None req-86b434d6-80e3-4e49-84cb-cc818ddb9cb5 f76fce57975c427684f3251fb16daf16 01115f1ff5974b5286707ef8e625f86b - - default default] [instance: 3935c340-7521-4082-b140-8d453e792cb8] Failed to build and run instance: nova.exception.NoUniqueMatch: Multiple security groups found matching 'ceph-security-group'. Use an ID to be more specific.
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8] Traceback (most recent call last):
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/compute/manager.py", line 2632, in _build_and_run_instance
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     self.driver.spawn(context, instance, image_meta,
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/virt/libvirt/driver.py", line 4657, in spawn
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     xml = self._get_guest_xml(context, instance, network_info,
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/virt/libvirt/driver.py", line 7817, in _get_guest_xml
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     network_info_str = str(network_info)
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]                        ^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/model.py", line 620, in __str__
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     return self._sync_wrapper(fn, *args, **kwargs)
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/model.py", line 603, in _sync_wrapper
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     self.wait()
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/model.py", line 635, in wait
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     self[:] = self._gt.wait()
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]               ^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/eventlet/greenthread.py", line 224, in wait
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     return self._exit_event.wait()
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]            ^^^^^^^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/eventlet/event.py", line 131, in wait
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     current.throw(*self._exc)
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/eventlet/greenthread.py", line 264, in main
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     result = function(*args, **kwargs)
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]              ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/utils.py", line 664, in context_wrapper
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     return func(*args, **kwargs)
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]            ^^^^^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/compute/manager.py", line 2002, in _allocate_network_async
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     raise e
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/compute/manager.py", line 1980, in _allocate_network_async
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     nwinfo = self.network_api.allocate_for_instance(
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/neutron.py", line 1242, in allocate_for_instance
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     security_group_ids = self._process_security_groups(
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/neutron.py", line 913, in _process_security_groups
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     security_group_ids = self._get_security_group_ids(
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/neutron.py", line 847, in _get_security_group_ids
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     raise exception.NoUniqueMatch(
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8] nova.exception.NoUniqueMatch: Multiple security groups found matching 'ceph-security-group'. Use an ID to be more specific.
2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2105896

Title:
  Multiple security groups in a project cause VM build failures

Status in OpenStack Compute (nova):
  New

Bug description:
  
  A bug was introduced by https://review.opendev.org/c/openstack/nova/+/811521, in which duplicate security group names from a project [1] can cause VM build errors such as the one below [2] [3].

  This occurs when creating a vm with a different security (by name) or
  by using a uuid. In this example I used the uuid for the management
  sec-group, but the vm build failed due to two security groups being
  the same name in the project.

  
  [1]
  # openstack security group list --project admin
  +--------------------------------------+---------------------+---------------------------------+----------------------------------+------+
  | ID                                   | Name                | Description                     | Project                          | Tags |
  +--------------------------------------+---------------------+---------------------------------+----------------------------------+------+
  | 123139a5-1ff2-4818-bc73-7acf839ef49b | ceph-security-group | ceph-security-group             | 01115f1ff5974b5286707ef8e625f86b | []   |
  | 28e42ade-5ac2-412e-80c2-74139f4d04b5 | ceph-security-group | Security group for Ceph cluster | 01115f1ff5974b5286707ef8e625f86b | []   |
  | 52457b0d-9fd4-4b74-9fec-a17fdca5c86d | default             | Default security group          | 01115f1ff5974b5286707ef8e625f86b | []   |
  | 9507768e-1740-4233-8840-28d58cad87cc | management          | management                      | 01115f1ff5974b5286707ef8e625f86b | []   |
  +--------------------------------------+---------------------+---------------------------------+----------------------------------+------+

  
  [2]

  ## openstack server show 3935c340-7521-4082-b140-8d453e792cb8 -c fault -f json
  compute version 2.89 is not in supported versions: 2, 2.1
  {
    "fault": {
      "code": 500,
      "created": "2025-04-01T18:25:54Z",
      "message": "Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3935c340-7521-4082-b140-8d453e792cb8.",
      "details": "Traceback (most recent call last):\n  File \"/var/lib/kolla/venv/lib/python3.12/site-packages/nova/conductor/manager.py\", line 705, in build_instances\n    raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3935c340-7521-4082-b140-8d453e792cb8.\n"
    }
  }

  
  [3]
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [None req-86b434d6-80e3-4e49-84cb-cc818ddb9cb5 f76fce57975c427684f3251fb16daf16 01115f1ff5974b5286707ef8e625f86b - - default default] [instance: 3935c340-7521-4082-b140-8d453e792cb8] Failed to build and run instance: nova.exception.NoUniqueMatch: Multiple security groups found matching 'ceph-security-group'. Use an ID to be more specific.
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8] Traceback (most recent call last):
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/compute/manager.py", line 2632, in _build_and_run_instance
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     self.driver.spawn(context, instance, image_meta,
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/virt/libvirt/driver.py", line 4657, in spawn
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     xml = self._get_guest_xml(context, instance, network_info,
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/virt/libvirt/driver.py", line 7817, in _get_guest_xml
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     network_info_str = str(network_info)
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]                        ^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/model.py", line 620, in __str__
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     return self._sync_wrapper(fn, *args, **kwargs)
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/model.py", line 603, in _sync_wrapper
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     self.wait()
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/model.py", line 635, in wait
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     self[:] = self._gt.wait()
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]               ^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/eventlet/greenthread.py", line 224, in wait
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     return self._exit_event.wait()
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]            ^^^^^^^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/eventlet/event.py", line 131, in wait
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     current.throw(*self._exc)
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/eventlet/greenthread.py", line 264, in main
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     result = function(*args, **kwargs)
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]              ^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/utils.py", line 664, in context_wrapper
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     return func(*args, **kwargs)
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]            ^^^^^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/compute/manager.py", line 2002, in _allocate_network_async
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     raise e
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/compute/manager.py", line 1980, in _allocate_network_async
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     nwinfo = self.network_api.allocate_for_instance(
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/neutron.py", line 1242, in allocate_for_instance
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     security_group_ids = self._process_security_groups(
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/neutron.py", line 913, in _process_security_groups
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     security_group_ids = self._get_security_group_ids(
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]   File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/network/neutron.py", line 847, in _get_security_group_ids
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]     raise exception.NoUniqueMatch(
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8] nova.exception.NoUniqueMatch: Multiple security groups found matching 'ceph-security-group'. Use an ID to be more specific.
  2025-04-01 18:25:40.401 7 ERROR nova.compute.manager [instance: 3935c340-7521-4082-b140-8d453e792cb8]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2105896/+subscriptions