yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #82852
[Bug 1881881] [NEW] nova api return HTTP 500 error when interface attach failed due to lack of instance PCI slots
Public bug reported:
Description
===========
When no more pci slot are available for hot pluggable network interface,
nova api return HTTP 500 internal error which is not very helpfull from
client point of view.
It seems that nova catch all libvirt error and raise:
nova.exception.InterfaceAttachFailed: Failed to attach network adapter device to 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53
Should it be better to handle "libvirt.libvirtError: internal error: No more available PCI slots" As a exc.HTTPConflict() instead of webob.exc.HTTPInternalServerError() for example?
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/attach_interfaces.py#L182
Steps to reproduce
==================
On a fresh devstack master install
1) spawn one instance
2) create 30 network interfaces
for i in $(seq 2 30) ; do openstack port create --network private port-${i} ; done
3) Attach interface to isntances:
for i in $(seq 2 30) ; do openstack server add port 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53 port-${i} ; done
last attach ones should failed we the folling out put:
openstack server add port 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53 port-26
Failed to attach network adapter device to 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53 (HTTP 500) (Request-ID: req-03476f4a-97ac-483d-bf15-e9f0bda776d4)
in Logs:
Jun 03 09:02:06 alex-devstack nova-compute[28931]: ERROR nova.virt.libvirt.driver [instance: 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53] File "/usr/local/lib/python3.6/dist-packages/libvirt.py", line 593, in attachDeviceFlags
Jun 03 09:02:06 alex-devstack nova-compute[28931]: ERROR nova.virt.libvirt.driver [instance: 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
Jun 03 09:02:06 alex-devstack nova-compute[28931]: ERROR nova.virt.libvirt.driver [instance: 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53] libvirt.libvirtError: internal error: No more available PCI slots
...
Jun 03 09:02:22 alex-devstack nova-compute[28931]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 7574, in attach_interface
Jun 03 09:02:22 alex-devstack nova-compute[28931]: ERROR oslo_messaging.rpc.server instance_uuid=instance.uuid)
Jun 03 09:02:22 alex-devstack nova-compute[28931]: ERROR oslo_messaging.rpc.server nova.exception.InterfaceAttachFailed: Failed to attach network adapter device to 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53
Expected result
===============
Getting HTTP conflict or other explicit response indicating that we reach a limit.
Actual result
=============
Getting HTTP 500 internal error.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1881881
Title:
nova api return HTTP 500 error when interface attach failed due to
lack of instance PCI slots
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
When no more pci slot are available for hot pluggable network
interface, nova api return HTTP 500 internal error which is not very
helpfull from client point of view.
It seems that nova catch all libvirt error and raise:
nova.exception.InterfaceAttachFailed: Failed to attach network adapter device to 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53
Should it be better to handle "libvirt.libvirtError: internal error: No more available PCI slots" As a exc.HTTPConflict() instead of webob.exc.HTTPInternalServerError() for example?
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/attach_interfaces.py#L182
Steps to reproduce
==================
On a fresh devstack master install
1) spawn one instance
2) create 30 network interfaces
for i in $(seq 2 30) ; do openstack port create --network private port-${i} ; done
3) Attach interface to isntances:
for i in $(seq 2 30) ; do openstack server add port 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53 port-${i} ; done
last attach ones should failed we the folling out put:
openstack server add port 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53 port-26
Failed to attach network adapter device to 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53 (HTTP 500) (Request-ID: req-03476f4a-97ac-483d-bf15-e9f0bda776d4)
in Logs:
Jun 03 09:02:06 alex-devstack nova-compute[28931]: ERROR nova.virt.libvirt.driver [instance: 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53] File "/usr/local/lib/python3.6/dist-packages/libvirt.py", line 593, in attachDeviceFlags
Jun 03 09:02:06 alex-devstack nova-compute[28931]: ERROR nova.virt.libvirt.driver [instance: 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
Jun 03 09:02:06 alex-devstack nova-compute[28931]: ERROR nova.virt.libvirt.driver [instance: 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53] libvirt.libvirtError: internal error: No more available PCI slots
...
Jun 03 09:02:22 alex-devstack nova-compute[28931]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 7574, in attach_interface
Jun 03 09:02:22 alex-devstack nova-compute[28931]: ERROR oslo_messaging.rpc.server instance_uuid=instance.uuid)
Jun 03 09:02:22 alex-devstack nova-compute[28931]: ERROR oslo_messaging.rpc.server nova.exception.InterfaceAttachFailed: Failed to attach network adapter device to 0c6d2b7a-07d8-4f64-baa8-a6c05fb6ce53
Expected result
===============
Getting HTTP conflict or other explicit response indicating that we reach a limit.
Actual result
=============
Getting HTTP 500 internal error.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1881881/+subscriptions