yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

To summarize, during post-copy on the source host, nova activates the
port binding on the dest host. During post live migration on the source
host, nova refreshes the ports in the instance info cache and then calls
unplug_vifs in the virt driver, but the vif type now shows up as
'unbound' - and I'm not sure if that's a problem in neutron or not; why
would the vif type be unbound if we activated the dest host port
binding? Is that a bug in neutron? We can workaround this in nova by
getting the vif type off the migrate_data.vifs and using that during the
unplug call, but it seems like we shouldn't need to do this, and the
unbound vif type might be the race (from neutron?).

** Changed in: neutron
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1783917

Title:
  live migration fails with NovaException: Unsupported VIF type unbound
  convert '_nova_to_osvif_vif_unbound'

Status in neutron:
  Invalid
Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  Seen in the gate starting today:

  http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22Unsupported%20VIF%20type%20unbound%20convert%20'_nova_to_osvif_vif_unbound'%5C%22%20AND%20tags:screen-n-cpu.txt&from=7d

  My guess is that it's related to https://review.openstack.org/522537
  which began using the new neutron port binding API.

  Trace looks like this:

  http://logs.openstack.org/52/564452/30/check/nova-live-
  migration/d3172a4/logs/screen-n-cpu.txt.gz#_Jul_26_22_44_44_739885

  Jul 26 22:44:44.739885 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: WARNING nova.virt.libvirt.driver [None req-7c0f6655-5920-4894-b3d5-062ba4f8270e service nova] [instance: 582afae2-510c-4b73-b41d-1edc93e902a2] Error monitoring migration: Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound': NovaException: Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound'
  Jul 26 22:44:44.740334 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2] Traceback (most recent call last):
  Jul 26 22:44:44.740578 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7557, in _live_migration
  Jul 26 22:44:44.740839 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     finish_event, disk_paths)
  Jul 26 22:44:44.741086 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7421, in _live_migration_monitor
  Jul 26 22:44:44.741328 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     migrate_data)
  Jul 26 22:44:44.741562 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 79, in wrapped
  Jul 26 22:44:44.741785 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     function_name, call_dict, binary, tb)
  Jul 26 22:44:44.742008 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  Jul 26 22:44:44.742246 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     self.force_reraise()
  Jul 26 22:44:44.742485 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  Jul 26 22:44:44.742791 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     six.reraise(self.type_, self.value, self.tb)
  Jul 26 22:44:44.743117 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 69, in wrapped
  Jul 26 22:44:44.743332 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     return f(self, context, *args, **kw)
  Jul 26 22:44:44.743544 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/compute/manager.py", line 216, in decorated_function
  Jul 26 22:44:44.743758 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     kwargs['instance'], e, sys.exc_info())
  Jul 26 22:44:44.745586 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  Jul 26 22:44:44.747347 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     self.force_reraise()
  Jul 26 22:44:44.747790 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  Jul 26 22:44:44.748166 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     six.reraise(self.type_, self.value, self.tb)
  Jul 26 22:44:44.748507 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/compute/manager.py", line 204, in decorated_function
  Jul 26 22:44:44.748859 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     return function(self, context, *args, **kwargs)
  Jul 26 22:44:44.749207 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/compute/manager.py", line 6582, in _post_live_migration
  Jul 26 22:44:44.749626 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     network_info)
  Jul 26 22:44:44.750459 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7971, in post_live_migration_at_source
  Jul 26 22:44:44.751201 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     self.unplug_vifs(instance, network_info)
  Jul 26 22:44:44.751576 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 829, in unplug_vifs
  Jul 26 22:44:44.752054 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     self._unplug_vifs(instance, network_info, False)
  Jul 26 22:44:44.752517 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 823, in _unplug_vifs
  Jul 26 22:44:44.752885 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     self.vif_driver.unplug(instance, vif)
  Jul 26 22:44:44.753247 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/virt/libvirt/vif.py", line 928, in unplug
  Jul 26 22:44:44.753613 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     vif_obj = os_vif_util.nova_to_osvif_vif(vif)
  Jul 26 22:44:44.753983 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]   File "/opt/stack/new/nova/nova/network/os_vif_util.py", line 505, in nova_to_osvif_vif
  Jul 26 22:44:44.754411 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]     {'type': vif['type'], 'func': funcname})
  Jul 26 22:44:44.754752 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2] NovaException: Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound'
  Jul 26 22:44:44.755092 ubuntu-xenial-rax-dfw-0000999143 nova-compute[2823]: ERROR nova.virt.libvirt.driver [instance: 582afae2-510c-4b73-b41d-1edc93e902a2]

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1783917/+subscriptions