yahoo-eng-team team mailing list archive

[Qin Zhao <chaochin@xxxxxxxxx>]

Public bug reported:

When a VM with an attached iSCSI disk fails to migrate,  the rollback
methods does not detach the disk from target host. What happens is
_lookup_by_name() fails, since the VM does not exist on the target host.
In detach_volume(), it is supposed to print a warning based on the
correct error code being returned, instead of throwing the exception.
However,  this is not happening, because _lookup_by_name() throws an
InstanceNotFound exception, rather than a libvirt.libvirtError
exception. So we also need to catch InstanceNotFound exception, so that
detach_volume() can continue to execute as expected.

Here's the exception log that I have:

2014-05-16 16:30:22.328 41419 WARNING nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Detaching volume from unknown instance
2014-05-16 16:30:22.331 41419 ERROR nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Failed to detach volume 98a940e5-051f-4d0f-a8c7-859a5079d95e from /dev/vdb
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Traceback (most recent call last):
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4218, in _detach_volume
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]     encryption=encryption)
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1356, in detach_volume
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]     virt_dom = self._lookup_by_name(instance_name)
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3477, in _lookup_by_name
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]     raise exception.InstanceNotFound(instance_id=instance_name)
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] InstanceNotFound: Instance rhel65_113-3e1d0d56-00000002 could not be found.
2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: in-stable-icehouse libvirt

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1321082

Title:
  libvirt driver detach_volume fails after migration failure

Status in OpenStack Compute (Nova):
  New

Bug description:
  When a VM with an attached iSCSI disk fails to migrate,  the rollback
  methods does not detach the disk from target host. What happens is
  _lookup_by_name() fails, since the VM does not exist on the target
  host.  In detach_volume(), it is supposed to print a warning based on
  the correct error code being returned, instead of throwing the
  exception. However,  this is not happening, because _lookup_by_name()
  throws an InstanceNotFound exception, rather than a
  libvirt.libvirtError exception. So we also need to catch
  InstanceNotFound exception, so that detach_volume() can continue to
  execute as expected.

  Here's the exception log that I have:

  2014-05-16 16:30:22.328 41419 WARNING nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Detaching volume from unknown instance
  2014-05-16 16:30:22.331 41419 ERROR nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Failed to detach volume 98a940e5-051f-4d0f-a8c7-859a5079d95e from /dev/vdb
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Traceback (most recent call last):
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4218, in _detach_volume
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]     encryption=encryption)
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1356, in detach_volume
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]     virt_dom = self._lookup_by_name(instance_name)
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3477, in _lookup_by_name
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]     raise exception.InstanceNotFound(instance_id=instance_name)
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] InstanceNotFound: Instance rhel65_113-3e1d0d56-00000002 could not be found.
  2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1321082/+subscriptions