yahoo-eng-team team mailing list archive

[Adam Gandelman <1357599@xxxxxxxxxxxxxxxxxx>]

** Also affects: nova/icehouse
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1357599

Title:
  race condition with neutron in nova migrate code

Status in OpenStack Compute (Nova):
  Fix Committed
Status in OpenStack Compute (nova) icehouse series:
  New

Bug description:
  The tempest test that does a resize on the instance from time to time
  fails with a neutron virtual interface timeout error. The reason why
  this is occurring is because resize_instance() calls:

              disk_info = self.driver.migrate_disk_and_power_off(
                      context, instance, migration.dest_host,
                      instance_type, network_info,
                      block_device_info)

  which calls destory() which unplugs the vifs(). Then,

              self.driver.finish_migration(context, migration, instance,
                                           disk_info,
                                           network_info,
                                           image, resize_instance,
                                           block_device_info, power_on)

  is called which expects a vif_plugged event. Since this happens on the
  same host the neutron agent is able to detect that the vif was
  unplugged then plugged because it happens so fast.  To fix this we
  should check if we are migrating to the same host if we are we should
  not expect to get an event.

  8d1] Setting instance vm_state to ERROR
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1] Traceback (most recent call last):
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]   File "/opt/stack/new/nova/nova/compute/manager.py", line 3714, in finish_resize
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]     disk_info, image)
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]   File "/opt/stack/new/nova/nova/compute/manager.py", line 3682, in _finish_resize
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]     old_instance_type, sys_meta)
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]   File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 82, in __exit__
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]     six.reraise(self.type_, self.value, self.tb)
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]   File "/opt/stack/new/nova/nova/compute/manager.py", line 3677, in _finish_resize
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]     block_device_info, power_on)
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 5302, in finish_migration
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]     block_device_info, power_on)
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3792, in _create_domain_and_network
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1]     raise exception.VirtualInterfaceCreateException()
  2014-08-14 00:03:58.010 1276 TRACE nova.compute.manager [instance: dca468e4-d26f-4ae2-a522-7d02ef7c98d1] VirtualInterfaceCreateException: Virtual Interface creation failed

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1357599/+subscriptions