yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1555287] Re: Libvirt driver broken for non-disk-image backends

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sean Dague <sean@xxxxxxxxx>
Date: Fri, 16 Jun 2017 12:15:04 -0000
Reply-to: Bug 1555287 <1555287@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: nova/liberty
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1555287

Title:
  Libvirt driver broken for non-disk-image backends

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) kilo series:
  Fix Released
Status in OpenStack Compute (nova) liberty series:
  Fix Released

Bug description:
  Recently the ceph job (and any other configuration that doesn't use
  disk image as the backend storage) started failing like this:

  
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     incoming.message))
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/exception.py", line 110, in wrapped
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     payload)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/exception.py", line 89, in wrapped
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/compute/manager.py", line 359, in decorated_function
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     LOG.warning(msg, e, instance=instance)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/compute/manager.py", line 328, in decorated_function
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/compute/manager.py", line 409, in decorated_function
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/compute/manager.py", line 316, in decorated_function
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     migration.instance_uuid, exc_info=True)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/compute/manager.py", line 293, in decorated_function
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/compute/manager.py", line 387, in decorated_function
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/compute/manager.py", line 375, in decorated_function
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/compute/manager.py", line 3876, in resize_instance
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     timeout, retry_interval)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7237, in migrate_disk_and_power_off
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     shared_storage)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     self.force_reraise()
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7230, in migrate_disk_and_power_off
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher     host=dest, on_execute=on_execute,
  2016-03-09 14:47:29.102 17597 ERROR oslo_messaging.rpc.dispatcher UnboundLocalError: local variable 'on_execute' referenced before assignment

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1555287/+subscriptions
References

[Bug 1555287] [NEW] Libvirt driver broken for non-disk-image backends
From: Dan Smith, 2016-03-09