← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #35032

[Bug 1471726] [NEW] libvirt: blockCommit fails if domain is not running, for attached cinder volumes

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Using a devstack setup, fairly latest!

1) Create a cinder volume (used GlusterFS as the cinder backed) - cv1
2) Attach cv1  to vm1 (vm1 is a nova VM in running state)
3) Create 2 snapshots of vol1 using cinder snapshot-create ... cv1-snap1, cv1-snap2
4) Stop the nova vm vm1 (Note that cinder still reports the volume cv1 as 'in-use')
5) From cinder, delete cv1-snap1. Since cv1-snap1 is _Not_ the active file, nova tries to do a blockCommit and fails with excp below:

2015-07-06 09:33:00.479 ERROR oslo_messaging.rpc.dispatcher [req-695dd8c5-2722-4cf2-ab0a-583b0dacd388 nova service] Exception during message handling: Requested operation is not valid: domain is not running
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     return func(*args, **kwargs)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 89, in wrapped
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     payload)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 72, in wrapped
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 2954, in volume_snapshot_delete
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     snapshot_id, delete_info)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2024, in volume_snapshot_delete
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     base_file = delete_info['base_file']
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2017, in volume_snapshot_delete
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     "snapshots.") % ver
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2003, in _volume_snapshot_delete
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     # paths are maintained relative by qemu.
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 183, in doit
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     result = proxy_call(self._autowrap, f, *args, **kwargs)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     rv = execute(f, *args, **kwargs)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     six.reraise(c, e, tb)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     rv = meth(*args, **kwargs)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 642, in blockCommit
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     if ret == -1: raise libvirtError ('virDomainBlockCommit() failed', dom=self)
2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher libvirtError: Requested operation is not valid: domain is not running

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: blockcommit libvirt nova

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1471726

Title:
  libvirt: blockCommit fails if domain is not running, for attached
  cinder volumes

Status in OpenStack Compute (Nova):
  New

Bug description:
  Using a devstack setup, fairly latest!

  1) Create a cinder volume (used GlusterFS as the cinder backed) - cv1
  2) Attach cv1  to vm1 (vm1 is a nova VM in running state)
  3) Create 2 snapshots of vol1 using cinder snapshot-create ... cv1-snap1, cv1-snap2
  4) Stop the nova vm vm1 (Note that cinder still reports the volume cv1 as 'in-use')
  5) From cinder, delete cv1-snap1. Since cv1-snap1 is _Not_ the active file, nova tries to do a blockCommit and fails with excp below:

  2015-07-06 09:33:00.479 ERROR oslo_messaging.rpc.dispatcher [req-695dd8c5-2722-4cf2-ab0a-583b0dacd388 nova service] Exception during message handling: Requested operation is not valid: domain is not running
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     return func(*args, **kwargs)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 89, in wrapped
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     payload)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 72, in wrapped
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 2954, in volume_snapshot_delete
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     snapshot_id, delete_info)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2024, in volume_snapshot_delete
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     base_file = delete_info['base_file']
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2017, in volume_snapshot_delete
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     "snapshots.") % ver
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2003, in _volume_snapshot_delete
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     # paths are maintained relative by qemu.
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 183, in doit
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     rv = execute(f, *args, **kwargs)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     six.reraise(c, e, tb)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     rv = meth(*args, **kwargs)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 642, in blockCommit
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher     if ret == -1: raise libvirtError ('virDomainBlockCommit() failed', dom=self)
  2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher libvirtError: Requested operation is not valid: domain is not running

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1471726/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next