yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1884504] Re: nova assume ports with vnic_type=direct-physical use the libvirt interface element when processing network-vif-deleted events

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Balazs Gibizer <1884504@xxxxxxxxxxxxxxxxxx>
Date: Wed, 16 Sep 2020 15:14:52 -0000
Reply-to: Bug 1884504 <1884504@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
I confirm this issue exists in ussuri and older branches but not on
master any more.

** Also affects: nova/ussuri
   Importance: Undecided
       Status: New

** Changed in: nova
       Status: New => Invalid

** Changed in: nova/ussuri
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1884504

Title:
  nova assume ports with vnic_type=direct-physical use the libvirt
  interface element when processing network-vif-deleted events

Status in neutron:
  Opinion
Status in OpenStack Compute (nova):
  Invalid
Status in OpenStack Compute (nova) ussuri series:
  Confirmed

Bug description:
  High level description
  It should not be able to remove a port that it is attached to a VM. It is allowed to removed a port with with vnic_type=direct-physical. Exception raised:

  LibvirtConfigGuestHostdevPCI accessing a non existing attribute:
  mac_addr

  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server [req-55778a4d-bf76-43cf-8198-02111be843d3 a82e4764680d4805b04dbc843df5ad73 233cc3938ae44e6e9fb21eaaff091999 - default default] Exception during message handling: AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8104, in external_instance_event
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     event.tag)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8014, in _process_instance_vif_deleted_event
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     self.driver.detach_interface(context, instance, vif)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1807, in detach_interface
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     interface = guest.get_interface_by_cfg(cfg)
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 246, in get_interface_by_cfg
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server     if (interface.mac_addr == cfg.mac_addr and
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'
  2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server 

  
  Step-by-step reproduction steps
  1. Create a vm with a "physical direct" port and attach to a VM

  (venv) (overcloud) [stack@undercloud-0 ~]$ openstack port list | grep "50.0"
  | eeb2cc67-046c-4414-a8ea-2be93bdbc542 | tempest-port-smoke-1184295488                   | f8:f2:1e:03:9b:e6 | ip_address='50.0.0.110', subnet_id='a9d92b9b-e6e4-40fe-a358-e21921fdbd2d'     | ACTIVE |
  (venv) (overcloud) [stack@undercloud-0 ~]$ openstack port show eeb2cc67-046c-4414-a8ea-2be93bdbc542
  +-----------------------+----------------------------------------------------------------------------------+
  | Field                 | Value                                                                            |
  +-----------------------+----------------------------------------------------------------------------------+
  | admin_state_up        | UP                                                                               |
  | allowed_address_pairs |                                                                                  |
  | binding_host_id       | compute-1.localdomain                                                            |
  | binding_profile       | pci_slot='0000:05:00.3', pci_vendor_info='8086:1572', physical_network='sriov-2' |
  | binding_vif_details   | port_filter='False', vlan='128'                                                  |
  | binding_vif_type      | hostdev_physical                                                                 |
  | binding_vnic_type     | direct-physical                                                                  |
  | created_at            | 2020-05-07T16:08:05Z                                                             |
  | data_plane_status     | None                                                                             |
  | description           |                                                                                  |
  | device_id             | 2c274d1d-b8b4-4db6-baae-b701613cc4c1                                             |
  | device_owner          | compute:nova                                                                     |
  | dns_assignment        | None                                                                             |
  | dns_name              | None                                                                             |
  | extra_dhcp_opts       |                                                                                  |
  | fixed_ips             | ip_address='50.0.0.110', subnet_id='a9d92b9b-e6e4-40fe-a358-e21921fdbd2d'        |
  | id                    | eeb2cc67-046c-4414-a8ea-2be93bdbc542                                             |
  | ip_address            | None                                                                             |
  | mac_address           | f8:f2:1e:03:9b:e6                                                                |
  | name                  | tempest-port-smoke-1184295488                                                    |
  | network_id            | c63b5f88-5447-49ab-8783-3f87a45ec9c7                                             |
  | option_name           | None                                                                             |
  | option_value          | None                                                                             |
  | port_security_enabled | True                                                                             |
  | project_id            | c2f705909fd2481b8e2dc81d9f03b29d                                                 |
  | qos_policy_id         | None                                                                             |
  | revision_number       | 9                                                                                |
  | security_group_ids    | c230b0e7-623f-43ef-b7ac-6f40f6104736                                             |
  | status                | ACTIVE                                                                           |
  | subnet_id             | None                                                                             |
  | tags                  |                                                                                  |
  | trunk_details         | None                                                                             |
  | updated_at            | 2020-05-07T16:08:36Z                                                             |
  +-----------------------+----------------------------------------------------------------------------------+

  2. Delete the port
  (venv) (overcloud) [stack@undercloud-0 ~]$ openstack port delete eeb2cc67-046c-4414-a8ea-2be93bdbc542
  (venv) (overcloud) [stack@undercloud-0 ~]$ openstack port list | grep "50.0"

  Apart from the error in the log file, this port is removed from the list of ports even being attached to a vm
  containers/nova/nova-compute.log:2020-05-07 16:10:38.019 9 ERROR oslo_messaging.rpc.server [req-42741c25-f49b-4c3e-97ff-e5d1d6ec91d7 9c58c668079a413fb9d3e86a3b0a3a50 29f45c177bd8480a83e1037873e30a88 - default default] Exception during message handling: AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'
  containers/nova/nova-compute.log:2020-05-07 16:10:38.019 9 ERROR oslo_messaging.rpc.server AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'

  Expected output
  Port should not be removed as it is attached to a VM

  Actual output
  Port is removed from the port list but it continues attached to the VM. 

  Version
  Queens

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1884504/+subscriptions
References

[Bug 1884504] [NEW] nova assume ports with vnic_type=direct-physical use the libvirt interface element when processing network-vif-deleted events
From: Miguel Angel Nieto Jimenez, 2020-06-22