yahoo-eng-team team mailing list archive

[sean mooney <1885558@xxxxxxxxxxxxxxxxxx>]

** Changed in: nova
   Importance: Undecided => Medium

** Also affects: nova/train
   Importance: Undecided
       Status: New

** Also affects: nova/ussuri
   Importance: Undecided
       Status: New

** Changed in: nova/train
   Importance: Undecided => Medium

** Changed in: nova/train
       Status: New => Triaged

** Changed in: nova/ussuri
   Importance: Undecided => Medium

** Changed in: nova/ussuri
       Status: New => Triaged

** Tags added: live-migration pci

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1885558

Title:
  sriov:  instance with macvtap vnic_type live migration failed

Status in OpenStack Compute (nova):
  In Progress
Status in OpenStack Compute (nova) train series:
  Triaged
Status in OpenStack Compute (nova) ussuri series:
  Triaged

Bug description:

  Instance with the vnic_type macvtap port live migration failed.

  My env configuration follow the document:
  https://docs.openstack.org/neutron/latest/admin/config-sriov.html

  
  # VFs on source comptue
  84:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  84:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  84:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  84:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  84:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  84:11.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  84:11.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  84:11.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  84:11.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

  # VFs on dest compute
  81:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  81:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  81:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  81:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  81:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  81:11.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  81:11.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  81:11.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  81:11.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)


  # port create CLI
  openstack port create --network $net_id --vnic-type macvtap macvtap01

  # boot instance with macvtap port
  ova boot --flavor 2C4G --nic port-id=$(neutron port-show macvtap01 -f value -c id) --block-device source=image,id=${image},dest=volume,size=100,shutdown=preserve,bootindex=0 --availability-zone nova:ctrl01.srvio.dev vm01_ctrl_macvtap

  # live migration failed

  nova live-migration vm01_ctrl_macvtap

  Source compute node log:

  /var/log/nova-compute.log
  2020-06-28 03:51:52.446 806489 DEBUG nova.virt.libvirt.vif [-] vif_type=hw_veb instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2020-06-23T03:35:55Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='vm01_ctrl_macvtap',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(1),host='ctrl01.srvio.dev',hostname='vm01-ctrl-macvtap',id=91,image_ref='',info_cache=InstanceInfoCache,instance_type_id=1,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2020-06-23T03:36:15Z,launched_on='ctrl01.srvio.dev',locked=False,locked_by=None,memory_mb=4096,metadata={},migration_context=<?>,new_flavor=None,node='ctrl01.srvio.dev',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='b36d8472f55e4fe88f8af98fe2c0ad8c',ramdisk_id='',reservation_id='r-j7a6v3fv',root_device_name='/dev/vda',root_gb=50,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member,admin',image_base_image_ref='',image_container_format='bare',image_disk_format='raw',image_hw_qemu_guest_agent='yes',image_min_disk='50',image_min_ram='0',owner_project_name='admin',owner_user_name='admin'},tags=<?>,task_state='migrating',terminated_at=None,trusted_certs=<?>,updated_at=2020-06-23T03:36:15Z,user_data=None,user_id='fdaa86b5871f4b2bad4febd7cc934a20',uuid=34fbcc84-2cfe-431d-89ee-132b2c9b4ed2,vcpu_model=<?>,vcpus=2,vm_mode=None,vm_state='active') vif={"profile": {"pci_slot": "0000:81:13.3", "physical_network": "sriovnet01", "pci_vendor_info": "8086:10ed"}, "ovs_interfaceid": null, "preserve_on_delete": true, "network": {"bridge": null, "label": "sriovnet01", "meta": {"injected": false, "tunneled": false, "mtu": 1500, "physical_network": "sriovnet01", "tenant_id": "b36d8472f55e4fe88f8af98fe2c0ad8c"}, "id": "ee578372-583f-4715-8d39-e4dc4138619c", "subnets": [{"ips": [{"meta": {}, "type": "fixed", "version": 4, "address": "192.168.120.20", "floating_ips": []}], "version": 4, "meta": {"dhcp_server": "192.168.120.2"}, "dns": [], "routes": [], "cidr": "192.168.120.0/24", "gateway": {"meta": {}, "type": "gateway", "version": 4, "address": "192.168.120.1"}}]}, "devname": "tap0495e774-55", "qbh_params": null, "vnic_type": "macvtap", "meta": {}, "details": {"port_filter": false, "vlan": "1120"}, "address": "fa:16:3e:ec:2e:4c", "active": true, "type": "hw_veb", "id": "0495e774-553c-4585-85c0-37211857a4ba", "qbg_params": null} virt_type=kvm get_config /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:624
  2020-06-28 03:51:52.446 806489 DEBUG nova.network.os_vif_util [-] Converting VIF {"profile": {"pci_slot": "0000:81:13.3", "physical_network": "sriovnet01", "pci_vendor_info": "8086:10ed"}, "ovs_interfaceid": null, "preserve_on_delete": true, "network": {"bridge": null, "label": "sriovnet01", "meta": {"injected": false, "tunneled": false, "mtu": 1500, "physical_network": "sriovnet01", "tenant_id": "b36d8472f55e4fe88f8af98fe2c0ad8c"}, "id": "ee578372-583f-4715-8d39-e4dc4138619c", "subnets": [{"ips": [{"meta": {}, "type": "fixed", "version": 4, "address": "192.168.120.20", "floating_ips": []}], "version": 4, "meta": {"dhcp_server": "192.168.120.2"}, "dns": [], "routes": [], "cidr": "192.168.120.0/24", "gateway": {"meta": {}, "type": "gateway", "version": 4, "address": "192.168.120.1"}}]}, "devname": "tap0495e774-55", "qbh_params": null, "vnic_type": "macvtap", "meta": {}, "details": {"port_filter": false, "vlan": "1120"}, "address": "fa:16:3e:ec:2e:4c", "active": true, "type": "hw_veb", "id": "0495e774-553c-4585-85c0-37211857a4ba", "qbg_params": null} nova_to_osvif_vif /usr/lib/python2.7/site-packages/nova/network/os_vif_util.py:523
  2020-06-28 03:51:52.447 806489 DEBUG nova.network.os_vif_util [-] No conversion for VIF type hw_veb yet nova_to_osvif_vif /usr/lib/python2.7/site-packages/nova/network/os_vif_util.py:539
  2020-06-28 03:51:52.447 806489 ERROR nova.virt.libvirt.driver [-] [instance: 34fbcc84-2cfe-431d-89ee-132b2c9b4ed2] Live Migration failure: PCI device 0000:81:13.3 not found: PciDeviceNotFoundById: PCI device 0000:81:13.3 not found

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1885558/+subscriptions