yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #76178
[Bug 1724628] Re: nova.virt.libvirt.driver failed to attach volume (RBD backed Cinder)
[Expired for OpenStack Compute (nova) because there has been no activity
for 60 days.]
** Changed in: nova
Status: Incomplete => Expired
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1724628
Title:
nova.virt.libvirt.driver failed to attach volume (RBD backed Cinder)
Status in OpenStack Compute (nova):
Expired
Bug description:
Hi,
I'm using Ubuntu Openstack Ocata deployed with Ubuntu Landscape. I
have installed three timees having the same problem with fresh
installation. Ceph is used for block and object storage.
When I try to attach an existing volume to an already running instance
it is not attached. Below is the nova-compute log running the machine:
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Failed to attach volume at mountpoint: /dev/vdb
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] guest.attach_device(conf, persistent=True, live=live)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] self._domain.attachDeviceFlags(device_xml, flags=flags)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] rv = execute(f, *args, **kwargs)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] six.reraise(c, e, tb)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] rv = meth(*args, **kwargs)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
2017-10-18 16:52:44.159 231733 ERROR nova.virt.libvirt.driver [instance: 17798414-d179-4195-9200-9b92790c5729]
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Driver failed to attach volume 33a23bbe-c997-4447-a497-0109773edcf8 at /dev/vdb
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] device_type=self['device_type'], encryption=encryption)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] self._disconnect_volume(connection_info, disk_dev)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] self.force_reraise()
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] guest.attach_device(conf, persistent=True, live=live)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] self._domain.attachDeviceFlags(device_xml, flags=flags)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] rv = execute(f, *args, **kwargs)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] six.reraise(c, e, tb)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] rv = meth(*args, **kwargs)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
2017-10-18 16:52:44.166 231733 ERROR nova.virt.block_device [instance: 17798414-d179-4195-9200-9b92790c5729]
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Failed to attach 33a23bbe-c997-4447-a497-0109773edcf8 at /dev/vdb
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] Traceback (most recent call last):
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4801, in _attach_volume
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] do_check_attach=False, do_driver_attach=True)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 48, in wrapped
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] ret_val = method(obj, context, *args, **kwargs)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 282, in attach
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] connector)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] self.force_reraise()
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] device_type=self['device_type'], encryption=encryption)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] self._disconnect_volume(connection_info, disk_dev)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] self.force_reraise()
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] guest.attach_device(conf, persistent=True, live=live)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] self._domain.attachDeviceFlags(device_xml, flags=flags)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] rv = execute(f, *args, **kwargs)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] six.reraise(c, e, tb)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] rv = meth(*args, **kwargs)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
2017-10-18 16:52:45.281 231733 ERROR nova.compute.manager [instance: 17798414-d179-4195-9200-9b92790c5729]
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server [req-5153c34c-8086-4a42-ac27-df6cc711a3bb 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] Exception during message handling
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 75, in wrapped
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server function_name, call_dict, binary)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 66, in wrapped
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 216, in decorated_function
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info())
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 204, in decorated_function
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4791, in attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server do_attach_volume(context, instance, driver_bdm)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 274, in inner
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4789, in do_attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server bdm.destroy()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4786, in do_attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server return self._attach_volume(context, instance, driver_bdm)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4809, in _attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self.volume_api.unreserve_volume(context, bdm.volume_id)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4801, in _attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server do_check_attach=False, do_driver_attach=True)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 48, in wrapped
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server ret_val = method(obj, context, *args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 282, in attach
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server connector)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 273, in attach
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server device_type=self['device_type'], encryption=encryption)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1243, in attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self._disconnect_volume(connection_info, disk_dev)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1232, in attach_volume
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server guest.attach_device(conf, persistent=True, live=live)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 309, in attach_device
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server self._domain.attachDeviceFlags(device_xml, flags=flags)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server rv = execute(f, *args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server six.reraise(c, e, tb)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server rv = meth(*args, **kwargs)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
2017-10-18 16:52:49.263 231733 ERROR oslo_messaging.rpc.server
If the instance is shutoff i can attach the volume, but when I try start the machine it can't.
The nova-compute log is:
2017-10-18 23:16:51.622 353512 WARNING nova.virt.libvirt.driver [req-0898e619-dfd5-4b17-9a13-5160c50b2161 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] USB tablet requested for guests by host configuration. In order to accept this request VNC should be enabled or SPICE and SPICE agent disabled on host.
2017-10-18 23:16:53.789 353512 ERROR nova.virt.libvirt.guest [req-0898e619-dfd5-4b17-9a13-5160c50b2161 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] Error launching a defined domain with XML: <domain type='kvm'>
<name>instance-00000001</name>
<uuid>17798414-d179-4195-9200-9b92790c5729</uuid>
<metadata>
<nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0">
<nova:package version="15.0.6"/>
<nova:name>vm-test01</nova:name>
<nova:creationTime>2017-10-18 23:16:51</nova:creationTime>
<nova:flavor name="m1.small">
<nova:memory>2048</nova:memory>
<nova:disk>20</nova:disk>
<nova:swap>0</nova:swap>
<nova:ephemeral>0</nova:ephemeral>
<nova:vcpus>1</nova:vcpus>
</nova:flavor>
<nova:owner>
<nova:user uuid="57646279ee6f40eb98ea3ee233d66875">admin</nova:user>
<nova:project uuid="d0aac06e67924d879543c2c3a5eb8a73">admin</nova:project>
</nova:owner>
<nova:root type="image" uuid="3176abd5-91f9-42f8-b7d4-b2125b91e54a"/>
</nova:instance>
</metadata>
<memory unit='KiB'>2097152</memory>
<currentMemory unit='KiB'>2097152</currentMemory>
<vcpu placement='static'>1</vcpu>
<cputune>
<shares>1024</shares>
</cputune>
<sysinfo type='smbios'>
<system>
<entry name='manufacturer'>OpenStack Foundation</entry>
<entry name='product'>OpenStack Nova</entry>
<entry name='version'>15.0.6</entry>
<entry name='serial'>1727c1d0-42f5-4837-8009-60317d3163c6</entry>
<entry name='uuid'>17798414-d179-4195-9200-9b92790c5729</entry>
<entry name='family'>Virtual Machine</entry>
</system>
</sysinfo>
<os>
<type arch='x86_64' machine='pc-i440fx-zesty'>hvm</type>
<boot dev='hd'/>
<smbios mode='sysinfo'/>
</os>
<features>
<acpi/>
<apic/>
</features>
<cpu mode='host-model'>
<model fallback='allow'/>
<topology sockets='1' cores='1' threads='1'/>
</cpu>
<clock offset='utc'>
<timer name='pit' tickpolicy='delay'/>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/kvm-spice</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/nova/instances/17798414-d179-4195-9200-9b92790c5729/disk'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<auth username='cinder'>
<secret type='ceph' uuid='514c9fca-8cbe-11e2-9c52-3bc8c7819472'/>
</auth>
<source protocol='rbd' name='cinder/volume-33a23bbe-c997-4447-a497-0109773edcf8'>
<host name='10.1.3.206' port='6789'/>
<host name='10.1.3.209' port='6789'/>
<host name='10.1.3.216' port='6789'/>
</source>
<target dev='vdb' bus='virtio'/>
<serial>33a23bbe-c997-4447-a497-0109773edcf8</serial>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>
<controller type='usb' index='0' model='piix3-uhci'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<interface type='bridge'>
<mac address='fa:16:3e:dc:e2:b7'/>
<source bridge='qbr25a13e1a-72'/>
<target dev='tap25a13e1a-72'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</interface>
<serial type='pty'>
<log file='/var/lib/nova/instances/17798414-d179-4195-9200-9b92790c5729/console.log' append='off'/>
<target port='0'/>
</serial>
<console type='pty'>
<log file='/var/lib/nova/instances/17798414-d179-4195-9200-9b92790c5729/console.log' append='off'/>
<target type='serial' port='0'/>
</console>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<memballoon model='virtio'>
<stats period='10'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</memballoon>
</devices>
</domain>
2017-10-18 23:16:53.791 353512 ERROR nova.virt.libvirt.driver [req-0898e619-dfd5-4b17-9a13-5160c50b2161 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Failed to start libvirt guest
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server [req-0898e619-dfd5-4b17-9a13-5160c50b2161 57646279ee6f40eb98ea3ee233d66875 d0aac06e67924d879543c2c3a5eb8a73 - - -] Exception during message handling
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 75, in wrapped
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server function_name, call_dict, binary)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 66, in wrapped
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 188, in decorated_function
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server LOG.warning(msg, e, instance=instance)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 157, in decorated_function
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 686, in decorated_function
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 216, in decorated_function
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info())
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 204, in decorated_function
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2535, in start_instance
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server self._power_on(context, instance)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2505, in _power_on
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server block_device_info)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2533, in power_on
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server self._hard_reboot(context, instance, network_info, block_device_info)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2422, in _hard_reboot
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server vifs_already_plugged=True)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5162, in _create_domain_and_network
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server destroy_disks_on_failure)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5134, in _create_domain_and_network
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server post_xml_callback=post_xml_callback)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5052, in _create_domain
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server guest.launch(pause=pause)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 145, in launch
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server self._encoded_xml, errors='ignore')
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 140, in launch
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server return self._domain.createWithFlags(flags)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server rv = execute(f, *args, **kwargs)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server six.reraise(c, e, tb)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server rv = meth(*args, **kwargs)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1065, in createWithFlags
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server libvirtError: internal error: process exited while connecting to monitor: 2017-10-18T23:16:53.331734Z qemu-system-x86_64: -drive file=rbd:cinder/volume-33a23bbe-c997-4447-a497-0109773edcf8:id=cinder:key=AQCmfOdZZJQELRAA1YtClJ40OMbJSd7NEtNlMA==:auth_supported=cephx\;none:mon_host=10.1.3.206\:6789\;10.1.3.209\:6789\;10.1.3.216\:6789,format=raw,if=none,id=drive-virtio-disk1,serial=33a23bbe-c997-4447-a497-0109773edcf8,cache=none: error connecting: Operation not permitted
2017-10-18 23:16:54.708 353512 ERROR oslo_messaging.rpc.server
2017-10-18 23:16:57.464 353512 WARNING nova.compute.manager [req-23580d43-ab30-4075-b80b-5204a7988415 4f4982450d084668a9f863e7b4c18ee1 60fe2149f9374a5a831a77812d647670 - - -] [instance: 17798414-d179-4195-9200-9b92790c5729] Received unexpected event network-vif-unplugged-25a13e1a-7283-46d5-bef5-2afa02010667 for instance
2017-10-18 23:19:23.129 353512 WARNING nova.compute.manager [req-276a5572-c7fc-48f0-9fc4-384f58fade7a - - - - -] While synchronizing instance power states, found 1 instances in the database and 0 instances on the hypervisor.
Regards,
Jaime Guzman
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1724628/+subscriptions
References