yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1476114] Re: Launch instance failed using instances' snapshot created volume

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Zhenyu Zheng <zhengzhenyu@xxxxxxxxxx>
Date: Wed, 17 May 2017 06:27:13 -0000
Reply-to: Bug 1476114 <1476114@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Also affects: cinder
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1476114

Title:
  Launch instance failed using instances' snapshot created volume

Status in Cinder:
  New
Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  Launching instance fails when using a volume that is created using a
  snapshot of  a volume-backended instance.

  How to reproduce:

  Step 1:
  Create an volume backended instance.

  root@zheng-dev1:/var/log/nova# nova boot --flavor 1 --boot-volume
  daaddb77-4257-4ccd-86f2-220b31a0ce9b --nic net-id=8744ee96-7690-43bb-
  89b4-fcac805557bc test1

  root@zheng-dev1:/var/log/nova# nova list
  +--------------------------------------+-------+--------+------------+-------------+--------------------------------+
  | ID                                   | Name  | Status | Task State | Power State | Networks                       |
  +--------------------------------------+-------+--------+------------+-------------+--------------------------------+
  | ef3c6074-4d38-4d7b-8d93-d0ace58d3a6a | test1 | ACTIVE | -          | Running     | public=2001:db8::6, 172.24.4.5 |
  +--------------------------------------+-------+--------+------------+-------------+--------------------------------+

  Step 2:
  Create a snapshot of this instance using nova image-create, this will create an image in glance.

  root@zheng-dev1:/var/log/nova# nova image-create ef3c6074-4d38-4d7b-8d93-d0ace58d3a6a test-image
  root@zheng-dev1:/var/log/nova# glance image-list
  +--------------------------------------+---------------------------------+-------------+------------------+----------+--------+
  | ID                                   | Name                            | Disk Format | Container Format | Size     | Status |
  +--------------------------------------+---------------------------------+-------------+------------------+----------+--------+
  | 7bdff9a3-d051-4e75-bcd3-de69dbffe063 | cirros-0.3.4-x86_64-uec         | ami         | ami              | 25165824 | active |
  | 2af2dce2-f778-4d73-b827-5281741fc1cf | cirros-0.3.4-x86_64-uec-kernel  | aki         | aki              | 4979632  | active |
  | 60ea7020-fcc1-4535-af5e-0e894a01a44a | cirros-0.3.4-x86_64-uec-ramdisk | ari         | ari              | 3740163  | active |
  | ce7b2d17-196a-4871-bc1b-9dcb184863be | test-image                      |             |                  |          | active |
  +--------------------------------------+---------------------------------+-------------+------------------+----------+--------+

  Step 3:
  Create a new volume using the previously created image.

  root@zheng-dev1:/var/log/nova# cinder create --image-id ce7b2d17-196a-4871-bc1b-9dcb184863be --name test-volume 1
  +---------------------------------------+--------------------------------------+
  |                Property               |                Value                 |
  +---------------------------------------+--------------------------------------+
  |              attachments              |                  []                  |
  |           availability_zone           |                 nova                 |
  |                bootable               |                false                 |
  |          consistencygroup_id          |                 None                 |
  |               created_at              |      2015-07-20T06:44:41.000000      |
  |              description              |                 None                 |
  |               encrypted               |                False                 |
  |                   id                  | cc21dc7d-aa4b-4e24-8f11-8b916c5d6347 |
  |                metadata               |                  {}                  |
  |              multiattach              |                False                 |
  |                  name                 |             test-volume              |
  |         os-vol-host-attr:host         |                 None                 |
  |     os-vol-mig-status-attr:migstat    |                 None                 |
  |     os-vol-mig-status-attr:name_id    |                 None                 |
  |      os-vol-tenant-attr:tenant_id     |   b8112a8d8227490eba99419b8a8c2555   |
  |   os-volume-replication:driver_data   |                 None                 |
  | os-volume-replication:extended_status |                 None                 |
  |           replication_status          |               disabled               |
  |                  size                 |                  1                   |
  |              snapshot_id              |                 None                 |
  |              source_volid             |                 None                 |
  |                 status                |               creating               |
  |                user_id                |   ed64bccd0227444fa02dbd7695769a7d   |
  |              volume_type              |             lvmdriver-1              |
  +---------------------------------------+--------------------------------------+
  root@zheng-dev1:/var/log/nova# cinder list
  +--------------------------------------+-----------+-----------------+------+-------------+----------+--------------------------------------+
  |                  ID                  |   Status  |       Name      | Size | Volume Type | Bootable |             Attached to              |
  +--------------------------------------+-----------+-----------------+------+-------------+----------+--------------------------------------+
  | cc21dc7d-aa4b-4e24-8f11-8b916c5d6347 | available |   test-volume   |  1   | lvmdriver-1 |   true   |                                      |
  | daaddb77-4257-4ccd-86f2-220b31a0ce9b |   in-use  | bootable-volume |  1   | lvmdriver-1 |   true   | ef3c6074-4d38-4d7b-8d93-d0ace58d3a6a |
  +--------------------------------------+-----------+-----------------+------+-------------+----------+--------------------------------------+

  Step 4:
  Using this volume to boot an new instance.

  root@zheng-dev1:/var/log/nova# nova boot --flavor 1 --boot-volume
  cc21dc7d-aa4b-4e24-8f11-8b916c5d6347 --nic net-id=8744ee96-7690-43bb-
  89b4-fcac805557bc test2


  ERROR LOG:

  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack [req-079d1169-0bbb-4b0a-bb1b-a72467585385 ed64bccd0227444fa02dbd7695769a7d b8112a8d8227490eba99419b8a8c2555 - - -] Caught error: 'unicode' object has no attribute 'update'
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack Traceback (most recent call last):
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/__init__.py", line 126, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     return req.get_response(self.application)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1317, in send
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     application, catch_exc_info=False)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1281, in call_application
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     return resp(environ, start_response)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 536, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     response = request.get_response(self._app)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1317, in send
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     application, catch_exc_info=False)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1281, in call_application
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     return resp(environ, start_response)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     return resp(environ, start_response)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 136, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     response = self.app(environ, start_response)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     return resp(environ, start_response)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 756, in __call__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     content_type, body, accept)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 821, in _process_stack
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 911, in dispatch
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     return method(req=request, **action_args)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 643, in create
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     self._handle_create_exception(*sys.exc_info())
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 472, in _handle_create_exception
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     six.reraise(*exc_info)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 628, in create
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     check_server_group_quota=check_server_group_quota)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/hooks.py", line 149, in inner
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     rv = f(*args, **kwargs)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/compute/api.py", line 1479, in create
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     check_server_group_quota=check_server_group_quota)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/compute/api.py", line 1114, in _create_instance
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     block_device_mapping, legacy_bdm)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/compute/api.py", line 738, in _check_and_transform_bdm
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     base_options, instance_type, image_meta, root_device_name)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/compute/api.py", line 684, in _get_image_defined_bdms
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     image_defined_bdms = image_properties.get('block_device_mapping', [])
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack   File "/opt/stack/nova/nova/block_device.py", line 83, in __init__
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack     bdm_dict.update(kwargs)
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack AttributeError: 'unicode' object has no attribute 'update'
  2015-07-20 14:45:27.450 12962 ERROR nova.api.openstack 
  2015-07-20 14:45:27.455 12962 INFO nova.api.openstack [req-079d1169-0bbb-4b0a-bb1b-a72467585385 ed64bccd0227444fa02dbd7695769a7d b8112a8d8227490eba99419b8a8c2555 - - -] http://10.250.10.17:8774/v2/b8112a8d8227490eba99419b8a8c2555/os-volumes_boot returned with HTTP 500

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1476114/+subscriptions
References

[Bug 1476114] [NEW] Launch instance failed using instances' snapshot created volume
From: Zhenyu Zheng, 2015-07-20