yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1597317] Re: Launch instance by using default lvm backend volume failed

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: OpenStack Infra <1597317@xxxxxxxxxxxxxxxxxx>
Date: Sat, 09 Jul 2016 03:56:13 -0000
Reply-to: Bug 1597317 <1597317@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Reviewed:  https://review.openstack.org/337081
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=f9d52850e7cd092dc736a77de366248c6dee6881
Submitter: Jenkins
Branch:    master

commit f9d52850e7cd092dc736a77de366248c6dee6881
Author: Cao Shufeng <caosf.fnst@xxxxxxxxxxxxxx>
Date:   Mon Jul 4 06:41:10 2016 -0400

    Do not conjecture volume-id from iscsi_name
    
    Now iscsi target driver assumes that volume name always start with
    "volume-". In fact the name can be configured[1]. This change gets
    the volume id from volume object directly.
    
    Closes-Bug: #1597317
    
    [1]: https://github.com/openstack/cinder/blob/9da9ebb34581818c053e1e92bf6f552e32605479/cinder/objects/volume.py#L142
    
    Change-Id: Iaa366fbc4ddc0265255e5a4d2bb9d166a665856c


** Changed in: cinder
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1597317

Title:
  Launch instance by using default lvm backend volume failed

Status in Cinder:
  Fix Released
Status in OpenStack Compute (nova):
  Incomplete

Bug description:
  
  Installed OpenStack Mitaka release (openstack 2.2.0)

  Default hypervisor libvirt+KVM is used
  Default volume backend "LVM" is used

  
  Steps followed

  1. Create bootable volume by using image "cirros" - OK
  2. Launch VM instance using the volume - NOK

  /var/log/cinder/volume.log
  ----------------------------

  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher [req-a3c3015e-353c-49d6-af28-8e97010a588b 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] Exception during message handling: list index out of range
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher     incoming.message))
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1441, in initialize_connection
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher     volume, connector)
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py", line 760, in create_export
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher     volume_path)
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py", line 195, in create_export
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher     chap_auth = self._get_target_chap_auth(context, iscsi_name)
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py", line 328, in _get_target_chap_auth
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher     vol_id = iscsi_name.split(':volume-')[1]
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher IndexError: list index out of range
  2016-06-29 07:45:22.278 24205 ERROR oslo_messaging.rpc.dispatcher
  2016-06-29 07:45:22.280 24205 ERROR oslo_messaging._drivers.common [req-a3c3015e-353c-49d6-af28-8e97010a588b 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] Returning exception list index out of range to caller
  2016-06-29 07:45:22.281 24205 ERROR oslo_messaging._drivers.common [req-a3c3015e-353c-49d6-af28-8e97010a588b 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n    incoming.message))\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1441, in initialize_connection\n    volume, connector)\n', '  File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py", line 760, in create_export\n    volume_path)\n', '  File "/usr/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py", line 195, in create_export\n    chap_auth = self._get_target_chap_auth(context, iscsi_name)\n', '  File "/usr/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py", line 328, in _get_target_chap_auth\n    vol_id = iscsi_name.split(\':volume-\')[1]\n', 'IndexError: list index out of range\n']
  2016-06-29 07:45:22.397 24205 ERROR cinder.volume.manager [req-864e09a5-1d40-49ce-b7c2-a46b1a91fe25 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] Terminate volume connection failed: 'NoneType' object has no attribute 'split'
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher [req-864e09a5-1d40-49ce-b7c2-a46b1a91fe25 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] Exception during message handling: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: 'NoneType' object has no attribute 'split'
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher     incoming.message))
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1540, in terminate_connection
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher     raise exception.VolumeBackendAPIException(data=err_msg)
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: 'NoneType' object has no attribute 'split'
  2016-06-29 07:45:22.398 24205 ERROR oslo_messaging.rpc.dispatcher
  2016-06-29 07:45:22.400 24205 ERROR oslo_messaging._drivers.common [req-864e09a5-1d40-49ce-b7c2-a46b1a91fe25 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] Returning exception Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: 'NoneType' object has no attribute 'split' to caller
  2016-06-29 07:45:22.401 24205 ERROR oslo_messaging._drivers.common [req-864e09a5-1d40-49ce-b7c2-a46b1a91fe25 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n    incoming.message))\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1540, in terminate_connection\n    raise exception.VolumeBackendAPIException(data=err_msg)\n', "VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: 'NoneType' object has no attribute 'split'\n"]


   /var/log/nova/nova-compute.log
  --------------------------------

  2016-06-29 07:44:40.014 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Auditing locally available compute resources for node hiqa-esx16.lab.nimblestorage.com
  2016-06-29 07:44:40.568 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Total usable vcpus: 24, total allocated vcpus: 11
  2016-06-29 07:44:40.568 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Final resource view: name=hiqa-esx16.lab.nimblestorage.com phys_ram=128739MB used_ram=23040MB phys_disk=449GB used_disk=220GB total_vcpus=24 used_vcpus=11 pci_stats=[]
  2016-06-29 07:44:40.622 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Compute_service record updated for hiqa-esx16.lab.nimblestorage.com:hiqa-esx16.lab.nimblestorage.com
  2016-06-29 07:45:20.594 5593 INFO nova.compute.claims [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Attempting claim: memory 2048 MB, disk 20 GB, vcpus 1 CPU
  2016-06-29 07:45:20.595 5593 INFO nova.compute.claims [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Total memory: 128739 MB, used: 23040.00 MB
  2016-06-29 07:45:20.596 5593 INFO nova.compute.claims [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] memory limit: 193108.50 MB, free: 170068.50 MB
  2016-06-29 07:45:20.596 5593 INFO nova.compute.claims [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Total disk: 449 GB, used: 220.00 GB
  2016-06-29 07:45:20.596 5593 INFO nova.compute.claims [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] disk limit not specified, defaulting to unlimited
  2016-06-29 07:45:20.597 5593 INFO nova.compute.claims [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Total vcpu: 24 VCPU, used: 11.00 VCPU
  2016-06-29 07:45:20.597 5593 INFO nova.compute.claims [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] vcpu limit: 384.00 VCPU, free: 373.00 VCPU
  2016-06-29 07:45:20.618 5593 INFO nova.compute.claims [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Claim successful
  2016-06-29 07:45:21.169 5593 WARNING nova.virt.libvirt.driver [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
  2016-06-29 07:45:21.171 5593 WARNING nova.virt.osinfo [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] Cannot find OS information - Reason: (No configuration information found for operating system Empty)
  2016-06-29 07:45:21.365 5593 INFO nova.virt.block_device [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Booting with blank volume at /dev/vda
  2016-06-29 07:45:22.292 5593 ERROR nova.volume.cinder [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] Initialize connection failed for volume 71d3efe0-236d-4904-8e0f-42b79f36a21d on host hiqa-esx16.lab.nimblestorage.com. Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-a3c3015e-353c-49d6-af28-8e97010a588b) Code: 500. Attempting to terminate connection.
  2016-06-29 07:45:22.408 5593 ERROR nova.volume.cinder [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] Connection between volume 71d3efe0-236d-4904-8e0f-42b79f36a21d and host hiqa-esx16.lab.nimblestorage.com might have succeeded, but attempt to terminate connection has failed. Validate the connection and determine if manual cleanup is needed. Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-864e09a5-1d40-49ce-b7c2-a46b1a91fe25) Code: 500.
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Instance failed block device setup
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Traceback (most recent call last):
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1754, in _prep_block_device
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     wait_func=self._await_block_device_map_created)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 518, in attach_block_devices
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     map(_log_and_attach, block_device_mapping)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 516, in _log_and_attach
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     bdm.attach(*attach_args, **attach_kwargs)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 54, in wrapped
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     ret_val = method(obj, context, *args, **kwargs)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 264, in attach
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     connector)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 232, in wrapper
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     res = method(self, ctx, *args, **kwargs)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 266, in wrapper
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     six.reraise(exc_value, None, exc_trace)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 259, in wrapper
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     res = method(self, ctx, volume_id, *args, **kwargs)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 437, in initialize_connection
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     'code': exc.code})
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     self.force_reraise()
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     six.reraise(self.type_, self.value, self.tb)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 412, in initialize_connection
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     context).volumes.initialize_connection(volume_id, connector)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/cinderclient/v2/volumes.py", line 379, in initialize_connection
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     {'connector': connector})
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/cinderclient/v2/volumes.py", line 309, in _action
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     resp, body = self.api.client.post(url, body=body)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 114, in post
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     return self._cs_request(url, 'POST', **kwargs)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 108, in _cs_request
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     return self.request(url, method, **kwargs)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 101, in request
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     raise exceptions.from_response(resp, body)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-a3c3015e-353c-49d6-af28-8e97010a588b)
  2016-06-29 07:45:22.475 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Build of instance 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2 aborted: Block Device Mapping is Invalid.
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Traceback (most recent call last):
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1926, in _do_build_and_run_instance
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     filter_properties)
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2083, in _build_and_run_instance
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     'create.error', fault=e)
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     self.force_reraise()
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     six.reraise(self.type_, self.value, self.tb)
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2048, in _build_and_run_instance
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     block_device_mapping) as resources:
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     return self.gen.next()
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2206, in _build_resources
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]     reason=e.format_message())
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] BuildAbortException: Build of instance 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2 aborted: Block Device Mapping is Invalid.
  2016-06-29 07:45:22.660 5593 ERROR nova.compute.manager [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2]
  2016-06-29 07:45:23.217 5593 INFO nova.compute.manager [req-e1febd8e-019d-4648-b1d0-4e4d93da6b97 46d26580334f4cb5b0cbfba58b926031 14bd05fdfb86490f90f8e72848833ba7 - - -] [instance: 0fbb161a-69a2-464a-a0b8-a8b9d11e8af2] Took 0.56 seconds to deallocate network for instance.
  2016-06-29 07:45:42.015 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Auditing locally available compute resources for node hiqa-esx16.lab.nimblestorage.com
  2016-06-29 07:45:42.586 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Total usable vcpus: 24, total allocated vcpus: 11
  2016-06-29 07:45:42.587 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Final resource view: name=hiqa-esx16.lab.nimblestorage.com phys_ram=128739MB used_ram=23040MB phys_disk=449GB used_disk=220GB total_vcpus=24 used_vcpus=11 pci_stats=[]
  2016-06-29 07:45:42.633 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Compute_service record updated for hiqa-esx16.lab.nimblestorage.com:hiqa-esx16.lab.nimblestorage.com
  2016-06-29 07:46:43.016 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Auditing locally available compute resources for node hiqa-esx16.lab.nimblestorage.com
  2016-06-29 07:46:43.575 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Total usable vcpus: 24, total allocated vcpus: 11
  2016-06-29 07:46:43.576 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Final resource view: name=hiqa-esx16.lab.nimblestorage.com phys_ram=128739MB used_ram=23040MB phys_disk=449GB used_disk=220GB total_vcpus=24 used_vcpus=11 pci_stats=[]
  2016-06-29 07:46:43.626 5593 INFO nova.compute.resource_tracker [req-528199e8-33aa-4e4b-a756-7b4cadf4ea98 - - - - -] Compute_service record updated for hiqa-esx16.lab.nimblestorage.com:hiqa-esx16.lab.nimblestorage.com

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1597317/+subscriptions
References

[Bug 1597317] [NEW] Launch instance by using default lvm backend volume failed
From: ganesan, 2016-06-29