yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1724628] [NEW] nova.virt.libvirt.driver failed to attach volume (RBD backed Cinder)

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Jaime Guzman <gokimax@xxxxxxxxx>
Date: Wed, 18 Oct 2017 17:16:45 -0000
Reply-to: Bug 1724628 <1724628@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Hi,

I'm using Ubuntu Openstack Ocata deployed with Ubuntu Landscape. I have
installed three timees having the same problem with fresh installation.
Ceph is used for block and object storage.

When I try to attach a volume to an instance this is not attached. Below
is the nova-compute log running the machine:

2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Failed to attach volume at mountpoint: /dev/vdb
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     guest.attach_device(conf, persistent=True, live=live)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     self._domain.attachDeviceFlags(device_xml, flags=flags)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = execute(f, *args, **kwargs)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(c, e, tb)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = meth(*args, **kwargs)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Driver failed to attach volume 33a23bbe-c997-4447-a497-0109773edcf8 at /dev/vdb
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     device_type=self['device_type'], encryption=encryption)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     self._disconnect_volume(connection_info, disk_dev)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     self.force_reraise()
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     guest.attach_device(conf, persistent=True, live=live)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     self._domain.attachDeviceFlags(device_xml, flags=flags)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = execute(f, *args, **kwargs)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(c, e, tb)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = meth(*args, **kwargs)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Failed to attach 33a23bbe-c997-4447-a497-0109773edcf8 at /dev/vdb
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4801, in _attach_volume
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     do_check_attach=False, do_driver_attach=True)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 48, in wrapped
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     ret_val = method(obj, context, *args, **kwargs)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 282, in attach
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     connector)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     self.force_reraise()
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     device_type=self['device_type'], encryption=encryption)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     self._disconnect_volume(connection_info, disk_dev)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     self.force_reraise()
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     guest.attach_device(conf, persistent=True, live=live)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     self._domain.attachDeviceFlags(device_xml, flags=flags)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = execute(f, *args, **kwargs)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(c, e, tb)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = meth(*args, **kwargs)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] Exception during message handling
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 75, in wrapped
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 66, in wrapped
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 216, in decorated_function
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 204, in decorated_function
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4791, in attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     do_attach_volume(context, instance, driver_bdm)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 274, in inner
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4789, in do_attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     bdm.destroy()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4786, in do_attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return self._attach_volume(context, instance, driver_bdm)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4809, in _attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.volume_api.unreserve_volume(context, bdm.volume_id)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4801, in _attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     do_check_attach=False, do_driver_attach=True)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 48, in wrapped
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     ret_val = method(obj, context, *args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 282, in attach
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     connector)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     device_type=self['device_type'], encryption=encryption)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self._disconnect_volume(connection_info, disk_dev)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     guest.attach_device(conf, persistent=True, live=live)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self._domain.attachDeviceFlags(device_xml, flags=flags)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     rv = execute(f, *args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(c, e, tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     rv = meth(*args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server

Regards,

Jaime Guzman

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1724628

Title:
  nova.virt.libvirt.driver failed to attach volume (RBD backed Cinder)

Status in OpenStack Compute (nova):
  New

Bug description:
  Hi,

  I'm using Ubuntu Openstack Ocata deployed with Ubuntu Landscape. I
  have installed three timees having the same problem with fresh
  installation. Ceph is used for block and object storage.

  When I try to attach a volume to an instance this is not attached.
  Below is the nova-compute log running the machine:

  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Failed to attach volume at mountpoint: /dev/vdb
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     guest.attach_device(conf, persistent=True, live=live)
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     self._domain.attachDeviceFlags(device_xml, flags=flags)
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = execute(f, *args, **kwargs)
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(c, e, tb)
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = meth(*args, **kwargs)
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
  2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Driver failed to attach volume 33a23bbe-c997-4447-a497-0109773edcf8 at /dev/vdb
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     device_type=self['device_type'], encryption=encryption)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     self._disconnect_volume(connection_info, disk_dev)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     self.force_reraise()
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     guest.attach_device(conf, persistent=True, live=live)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     self._domain.attachDeviceFlags(device_xml, flags=flags)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = execute(f, *args, **kwargs)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(c, e, tb)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = meth(*args, **kwargs)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
  2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Failed to attach 33a23bbe-c997-4447-a497-0109773edcf8 at /dev/vdb
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4801, in _attach_volume
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     do_check_attach=False, do_driver_attach=True)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 48, in wrapped
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     ret_val = method(obj, context, *args, **kwargs)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 282, in attach
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     connector)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     self.force_reraise()
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     device_type=self['device_type'], encryption=encryption)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     self._disconnect_volume(connection_info, disk_dev)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     self.force_reraise()
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     guest.attach_device(conf, persistent=True, live=live)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     self._domain.attachDeviceFlags(device_xml, flags=flags)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = execute(f, *args, **kwargs)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     six.reraise(c, e, tb)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     rv = meth(*args, **kwargs)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
  2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] Exception during message handling
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 75, in wrapped
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 66, in wrapped
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 216, in decorated_function
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 204, in decorated_function
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4791, in attach_volume
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     do_attach_volume(context, instance, driver_bdm)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 274, in inner
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4789, in do_attach_volume
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     bdm.destroy()
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4786, in do_attach_volume
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     return self._attach_volume(context, instance, driver_bdm)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4809, in _attach_volume
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.volume_api.unreserve_volume(context, bdm.volume_id)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4801, in _attach_volume
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     do_check_attach=False, do_driver_attach=True)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 48, in wrapped
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     ret_val = method(obj, context, *args, **kwargs)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 282, in attach
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     connector)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     device_type=self['device_type'], encryption=encryption)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self._disconnect_volume(connection_info, disk_dev)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     guest.attach_device(conf, persistent=True, live=live)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     self._domain.attachDeviceFlags(device_xml, flags=flags)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     rv = execute(f, *args, **kwargs)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     six.reraise(c, e, tb)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     rv = meth(*args, **kwargs)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
  2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server

  Regards,

  Jaime Guzman

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1724628/+subscriptions
Follow ups

[Bug 1724628] Re: nova.virt.libvirt.driver failed to attach volume (RBD backed Cinder)
From: Launchpad Bug Tracker, 2018-12-11