yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1261675] [NEW] Unable to snapshot an instance with ephemeral RBD

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Pavel Kirpichyov <pavel.kirpichyov@xxxxxxxxx>
Date: Tue, 17 Dec 2013 09:57:32 -0000
Reply-to: Bug 1261675 <1261675@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

When creating a snapshot from a VM instance, following error appears in
Nova compute log:

 2013-12-17 10:39:47.943 28210 ERROR nova.openstack.common.rpc.amqp [req-3a82cad9-3213-47e8-8a42-0f6e2a75008a a2eecc1caf5f40c5ab50405b68730c20 e255029e4a614ed1a5412192db588e74] Exception during message handling
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     **args)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 353, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 90, in wrapped
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     payload)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 73, in wrapped
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 243, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     pass
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 229, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 271, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     e, sys.exc_info())
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 258, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 319, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     % image_id, instance=instance)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     *args, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2293, in snapshot_instance
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     task_states.IMAGE_SNAPSHOT)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2324, in _snapshot_instance
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     update_task_state)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1399, in snapshot
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     snapshot_backend.snapshot_extract(out_path, image_format)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 531, in snapshot_extract
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     images.convert_image(snap, target, out_format)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 179, in convert_image
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     utils.execute(*cmd, run_as_root=run_as_root)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 177, in execute
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return processutils.execute(*cmd, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py", line 178, in execute
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     cmd=' '.join(cmd))
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command.
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Command: qemu-img convert -O qcow2 rbd:instances/instance-0000000f_disk /var/lib/nova/instances/snapshots/tmp63tXq0/b916d3ca7fbe46cba4ca7ce5e0138ee9
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Exit code: 1
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Stdout: ''
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Stderr: "qemu-img: error connecting\nqemu-img: Could not open 'rbd:instances/instance-0000000f_disk': Operation not supported\nqemu-img: Could not open 'rbd:instances/instance-0000000f_disk'\n"

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: rbd snapshot

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1261675

Title:
  Unable to snapshot an instance with ephemeral RBD

Status in OpenStack Compute (Nova):
  New

Bug description:
  When creating a snapshot from a VM instance, following error appears
  in Nova compute log:

   2013-12-17 10:39:47.943 28210 ERROR nova.openstack.common.rpc.amqp [req-3a82cad9-3213-47e8-8a42-0f6e2a75008a a2eecc1caf5f40c5ab50405b68730c20 e255029e4a614ed1a5412192db588e74] Exception during message handling
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     **args)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 353, in decorated_function
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 90, in wrapped
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     payload)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 73, in wrapped
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 243, in decorated_function
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     pass
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 229, in decorated_function
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 271, in decorated_function
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     e, sys.exc_info())
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 258, in decorated_function
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 319, in decorated_function
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     % image_id, instance=instance)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     *args, **kwargs)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2293, in snapshot_instance
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     task_states.IMAGE_SNAPSHOT)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2324, in _snapshot_instance
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     update_task_state)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1399, in snapshot
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     snapshot_backend.snapshot_extract(out_path, image_format)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 531, in snapshot_extract
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     images.convert_image(snap, target, out_format)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 179, in convert_image
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     utils.execute(*cmd, run_as_root=run_as_root)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 177, in execute
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     return processutils.execute(*cmd, **kwargs)
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py", line 178, in execute
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp     cmd=' '.join(cmd))
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command.
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Command: qemu-img convert -O qcow2 rbd:instances/instance-0000000f_disk /var/lib/nova/instances/snapshots/tmp63tXq0/b916d3ca7fbe46cba4ca7ce5e0138ee9
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Exit code: 1
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Stdout: ''
  2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Stderr: "qemu-img: error connecting\nqemu-img: Could not open 'rbd:instances/instance-0000000f_disk': Operation not supported\nqemu-img: Could not open 'rbd:instances/instance-0000000f_disk'\n"

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1261675/+subscriptions
Follow ups

[Bug 1261675] Re: Unable to snapshot an instance with ephemeral RBD
From: Thierry Carrez, 2014-01-22
[Bug 1261675] [NEW] Unable to snapshot an instance with ephemeral RBD
From: Pavel Kirpichyov, 2013-12-17
References

[Bug 1261675] [NEW] Unable to snapshot an instance with ephemeral RBD
From: Pavel Kirpichyov, 2013-12-17