yahoo-eng-team team mailing list archive

[Qin Zhao <chaochin@xxxxxxxxx>]

Public bug reported:

When I run longevity test against Juno code, I notice that that delete
VM operation occasionally fails. The stack trace is:

File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2507, in _delete_instance
    self._shutdown_instance(context, instance, bdms)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2437, in _shutdown_instance
    requested_networks)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2426, in _shutdown_instance
    block_device_info)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1054, in destroy
    self._destroy(instance)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1010, in _destroy
    instance=instance)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 979, in _destroy
    virt_dom.destroy()
  File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 183, in doit
    result = proxy_call(self._autowrap, f, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 141, in proxy_call
    rv = execute(f, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 122, in execute
    six.reraise(c, e, tb)
  File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 80, in tworker
    rv = meth(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 730, in destroy
    if ret == -1: raise libvirtError ('virDomainDestroy() failed', dom=self)


Libvirt log is:

2014-10-29 06:28:17.535+0000: 2025: warning : qemuProcessKill:4174 : Timed out waiting after SIGTERM to process 9972, sending SIGKILL
2014-10-29 06:28:22.537+0000: 2025: warning : qemuProcessKill:4206 : Timed out waiting after SIGKILL to process 9972
2014-10-29 06:28:22.537+0000: 2025: error : qemuDomainDestroyFlags:2098 : operation failed: failed to kill qemu process with SIGTERM

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: libvirt

** Bug watch added: Red Hat Bugzilla #1073624
   https://bugzilla.redhat.com/show_bug.cgi?id=1073624

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1387950

Title:
  libvirt: fail to delete VM due to libvirt timeout

Status in OpenStack Compute (Nova):
  New

Bug description:
  When I run longevity test against Juno code, I notice that that delete
  VM operation occasionally fails. The stack trace is:

  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2507, in _delete_instance
      self._shutdown_instance(context, instance, bdms)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2437, in _shutdown_instance
      requested_networks)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
      six.reraise(self.type_, self.value, self.tb)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2426, in _shutdown_instance
      block_device_info)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1054, in destroy
      self._destroy(instance)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1010, in _destroy
      instance=instance)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
      six.reraise(self.type_, self.value, self.tb)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 979, in _destroy
      virt_dom.destroy()
    File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 183, in doit
      result = proxy_call(self._autowrap, f, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 141, in proxy_call
      rv = execute(f, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 122, in execute
      six.reraise(c, e, tb)
    File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 80, in tworker
      rv = meth(*args, **kwargs)
    File "/usr/lib64/python2.6/site-packages/libvirt.py", line 730, in destroy
      if ret == -1: raise libvirtError ('virDomainDestroy() failed', dom=self)

  
  Libvirt log is:

  2014-10-29 06:28:17.535+0000: 2025: warning : qemuProcessKill:4174 : Timed out waiting after SIGTERM to process 9972, sending SIGKILL
  2014-10-29 06:28:22.537+0000: 2025: warning : qemuProcessKill:4206 : Timed out waiting after SIGKILL to process 9972
  2014-10-29 06:28:22.537+0000: 2025: error : qemuDomainDestroyFlags:2098 : operation failed: failed to kill qemu process with SIGTERM

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1387950/+subscriptions