← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1621076] Re: Can't detach interface from VM (if VM has two interface with same mac addresses)

 

Reviewed:  https://review.openstack.org/372243
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a5c38cc861f052b89bc1ce182f68784c493c723e
Submitter: Jenkins
Branch:    master

commit a5c38cc861f052b89bc1ce182f68784c493c723e
Author: Leehom Li (feli5) <feli5@xxxxxxxxx>
Date:   Mon Sep 19 13:43:46 2016 +0800

    Support detach interface with same MAC from instance
    
    When detach_interface nova uses <interface> XML generated
    from scatch, which is missing the PCI device information that
    libvirt would use to uniquely identify devices.
    
    In case instance has mutiple interfaces with same MAC address.
    Libvirt will failed with below error message:
        libvirtError: operation failed: multiple devices matching
        mac address fa:16:3e:60:46:1f found
    
    This patch fixes this problem by provide a new function
    get_interface_by_cfg, this function uses cfg generated by
    nova.virt.libvirt.vif.get_config  as parameter,
    return a LibvirtConfigGuestInterface object.
    
    Also added function format_dom for
    LibvirtConfigGuestDeviceAddressPCI, which will be used to
    generate pci address for LibvirtConfigGuestInterface.
    
    Change-Id: I8acae90c9d2111ed35f58f374f321d64f01ba563
    Closes-Bug: #1621076


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1621076

Title:
  Can't detach interface from VM (if VM has two interface with same mac
  addresses)

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  How to reproduce:

  1. Run any VM.
  2. Create two networks.
  3. Create two ports for each network with same mac addresses
  4. Attach those ports to VM
  5. Try to detach any interface.

  Expected result:
  The interface should be detached from VM.

  Actual result:
  We don't get any errors (via API) on previous steps but an interface still attached to VM

  Environment:
  * fuel_release: 9.0
  * fuel_openstack_version: mitaka-9.0
  * libvirt0: 1.2.9.3-9~u14.04+mos10
  * hypervisor: Libvirt + KVM


  Example:

  (OpenStack-venv)agent@laptop ~/projects $ nova interface-list c5ae5a9a-54a2-47b8-800e-619b8bc286f5
  +------------+--------------------------------------+--------------------------------------+----------------+-------------------+
  | Port State | Port ID                              | Net ID                               | IP addresses   | MAC Addr          |
  +------------+--------------------------------------+--------------------------------------+----------------+-------------------+
  | ACTIVE     | 0d9377b4-4f8f-467a-bae7-54d0d70e5262 | 3f5adcaa-d3e5-4caf-be8f-474751de5589 | 192.168.0.1    | fa:16:3e:60:46:1f |
  | ACTIVE     | 13cac036-7b6d-4188-879f-650c8d9e1f63 | 28939866-7379-4279-800c-b64c2776e1e0 | 192.168.111.82 | fa:16:3e:24:0d:a4 |
  | ACTIVE     | 310b9883-806d-4038-a095-1625abecbcb1 | 311c5a7e-5cb0-47e2-8aa0-20d74c4ee8c2 | 192.168.0.1    | fa:16:3e:60:46:1f |
  +------------+--------------------------------------+--------------------------------------+----------------+-------------------+
  (OpenStack-venv)agent@laptop ~/projects $ nova interface-detach c5ae5a9a-54a2-47b8-800e-619b8bc286f5 0d9377b4-4f8f-467a-bae7-54d0d70e5262
  (OpenStack-venv)agent@laptop ~/projects $ nova interface-list c5ae5a9a-54a2-47b8-800e-619b8bc286f5
  +------------+--------------------------------------+--------------------------------------+----------------+-------------------+
  | Port State | Port ID                              | Net ID                               | IP addresses   | MAC Addr          |
  +------------+--------------------------------------+--------------------------------------+----------------+-------------------+
  | ACTIVE     | 0d9377b4-4f8f-467a-bae7-54d0d70e5262 | 3f5adcaa-d3e5-4caf-be8f-474751de5589 | 192.168.0.1    | fa:16:3e:60:46:1f |
  | ACTIVE     | 13cac036-7b6d-4188-879f-650c8d9e1f63 | 28939866-7379-4279-800c-b64c2776e1e0 | 192.168.111.82 | fa:16:3e:24:0d:a4 |
  | ACTIVE     | 310b9883-806d-4038-a095-1625abecbcb1 | 311c5a7e-5cb0-47e2-8aa0-20d74c4ee8c2 | 192.168.0.1    | fa:16:3e:60:46:1f |
  +------------+--------------------------------------+--------------------------------------+----------------+-------------------+

  logs from compute:

  libvirt:
  <11>Sep  7 12:18:00 node-9 libvirtd: 11320: error : virDomainNetFindIdx:11005 : operation failed: multiple devices matching mac address fa:16:3e:60:46:1f found

  nova compute:
  <183>Sep  6 17:58:04 node-7 nova-compute: 2016-09-06 17:58:04.348 7652 DEBUG nova.objects.instance [req-db381757-3fbf-4bb8-a4df-160e1422a005 2b96e098d62147d8b9a15f635e0dd51a 4b867602d6974059afb2489a71dfaabb - - -] Lazy-loading 'flavor' on Instance uuid d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a obj_load_attr /usr/lib/python2.7/dist-packages/nova/objects/instance.py:895
  <183>Sep  6 17:58:04 node-7 nova-compute: 2016-09-06 17:58:04.430 7652 DEBUG nova.virt.libvirt.vif [req-db381757-3fbf-4bb8-a4df-160e1422a005 2b96e098d62147d8b9a15f635e0dd51a 4b867602d6974059afb2489a71dfaabb - - -] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone=None,cell_name=None,cleaned=False,config_drive='True',created_at=2016-09-06T17:41:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,disable_terminate=False,display_description='4894d01a-e464-4d17-b309-3c39a9875147',display_name='4894d01a-e464-4d17-b309-3c39a9875147',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(211),host='node-7.domain.tld',hostname='4894d01a-e464-4d17-b309-3c39a9875147',id=277,image_ref='9213a377-e2c2-4cc1-b1e8-483369db03fa',info_cache=InstanceInfoCache,instance_type_id=211,kernel_id='',key_data=None,key_name=None,launch_index=0,launched_at=2016-09-06T17:41:08Z,launched_on='node-7.domain.tld',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='node-7.domain.tld',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4b867602d6974059afb2489a71dfaabb',ramdisk_id='',reservation_id='r-ybcxxyxm',root_device_name='/dev/hda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={image_base_image_ref='9213a377-e2c2-4cc1-b1e8-483369db03fa',image_container_format='bare',image_disk_format='iso',image_min_disk='1',image_min_ram='0'},tags=<?>,task_state=None,terminated_at=None,updated_at=2016-09-06T17:41:08Z,user_data=None,user_id='2b96e098d62147d8b9a15f635e0dd51a',uuid=d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif=VIF({'profile': {}, 'ovs_interfaceid': u'4461eba1-a2cd-48f4-9289-86fc8d33bf12', 'preserve_on_delete': True, 'network': Network({'bridge': u'br-int', 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'versio
  <183>Sep  6 17:58:04 node-7 nova-compute: 2016-09-06 17:58:04.432 7652 DEBUG nova.virt.libvirt.vif [req-db381757-3fbf-4bb8-a4df-160e1422a005 2b96e098d62147d8b9a15f635e0dd51a 4b867602d6974059afb2489a71dfaabb - - -] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone=None,cell_name=None,cleaned=False,config_drive='True',created_at=2016-09-06T17:41:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,disable_terminate=False,display_description='4894d01a-e464-4d17-b309-3c39a9875147',display_name='4894d01a-e464-4d17-b309-3c39a9875147',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(211),host='node-7.domain.tld',hostname='4894d01a-e464-4d17-b309-3c39a9875147',id=277,image_ref='9213a377-e2c2-4cc1-b1e8-483369db03fa',info_cache=InstanceInfoCache,instance_type_id=211,kernel_id='',key_data=None,key_name=None,launch_index=0,launched_at=2016-09-06T17:41:08Z,launched_on='node-7.domain.tld',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='node-7.domain.tld',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4b867602d6974059afb2489a71dfaabb',ramdisk_id='',reservation_id='r-ybcxxyxm',root_device_name='/dev/hda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={image_base_image_ref='9213a377-e2c2-4cc1-b1e8-483369db03fa',image_container_format='bare',image_disk_format='iso',image_min_disk='1',image_min_ram='0'},tags=<?>,task_state=None,terminated_at=None,updated_at=2016-09-06T17:41:08Z,user_data=None,user_id='2b96e098d62147d8b9a15f635e0dd51a',uuid=d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif=VIF({'profile': {}, 'ovs_interfaceid': u'4461eba1-a2cd-48f4-9289-86fc8d33bf12', 'preserve_on_delete': True, 'network': Network({'bridge': u'br-int', 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'versio
  <183>Sep  6 17:58:04 node-7 nova-compute: 2016-09-06 17:58:04.436 7652 DEBUG nova.virt.libvirt.config [req-db381757-3fbf-4bb8-a4df-160e1422a005 2b96e098d62147d8b9a15f635e0dd51a 4b867602d6974059afb2489a71dfaabb - - -] Generated XML ('<interface type="bridge">\n  <mac address="fa:16:3e:74:b0:c8"/>\n  <model type="virtio"/>\n  <source bridge="br-int"/>\n  <target dev="tap4461eba1-a2"/>\n  <virtualport type="openvswitch">\n    <parameters interfaceid="4461eba1-a2cd-48f4-9289-86fc8d33bf12"/>\n  </virtualport>\n</interface>\n',)  to_xml /usr/lib/python2.7/dist-packages/nova/virt/libvirt/config.py:82
  <179>Sep  6 17:58:04 node-7 nova-compute: 2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [req-db381757-3fbf-4bb8-a4df-160e1422a005 2b96e098d62147d8b9a15f635e0dd51a 4b867602d6974059afb2489a71dfaabb - - -] [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a] detaching network adapter failed.
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a] Traceback (most recent call last):
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1539, in detach_interface
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]     guest.detach_device(cfg, persistent=True, live=live)
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 357, in detach_device
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]     self._domain.detachDeviceFlags(conf.to_xml(), flags=flags)
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]     rv = execute(f, *args, **kwargs)
  2016-09-06 17:58:04.441 7652 ERROR nova.virt.libvirt.driver [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a]   File "/usr/lib/python2.7/dist-packages/eventlet/
  <180>Sep  6 17:58:04 node-7 nova-compute: 2016-09-06 17:58:04.473 7652 WARNING nova.compute.manager [req-db381757-3fbf-4bb8-a4df-160e1422a005 2b96e098d62147d8b9a15f635e0dd51a 4b867602d6974059afb2489a71dfaabb - - -] [instance: d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a] Detach interface failed, port_id=4461eba1-a2cd-48f4-9289-86fc8d33bf12, reason: Failed to detach network adapter device from d95ab3d6-6b8f-42f0-8bf8-a553e99ee41a

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1621076/+subscriptions


References