yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1861501] [NEW] Store ID fetched from URI is incorrectly encoded under py27

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Alan Bishop <abishop@xxxxxxxxxx>
Date: Fri, 31 Jan 2020 14:24:15 -0000
Reply-to: Bug 1861501 <1861501@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Whenever a client updates the locations for an image, glance checks to
be sure the locations metadata includes the backend store ID that it
parses from the request URI. However, under py27 the parsed store ID
will not be properly encoded (it will be a str, but needs to be
unicode). This leads to a subsequent failure, when glance checks to
verify the value isinstance(val, six.text_type) at [1].

The problem can be triggered by having nova create an image from a
running VM. Under these circumstances, the store ID parsed by [2] and
added at [3] will be encoded incorrectly.

[1] https://opendev.org/openstack/glance_store/src/branch/master/glance_store/backend.py#L381
[2] https://opendev.org/openstack/glance/src/branch/master/glance/common/store_utils.py#L159
[3] https://opendev.org/openstack/glance/src/branch/master/glance/common/store_utils.py#L195

This leads to the following error:

2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi [req-81008491-28d6-442c-b95e-9b0d2f108213 11d8e44ae836459cbc32cd31d452f068 14efbf135f20447f8f45815b8dcb5249 - default default] Caught error: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.: BackendException: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi Traceback (most recent call last):
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1474, in __call__
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     request, **action_args)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1517, in dispatch
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     return method(*args, **kwargs)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 417, in wrapped
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     return func(self, req, *args, **kwargs)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 256, in update
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     change_method(req, image, change)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 309, in _do_add
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     self._do_add_locations(image, path[1], value)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 565, in _do_add_locations
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     image.locations.insert(pos, value)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 297, in _checker
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     return method(*args, **kwargs)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 292, in insert
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     return self.locations.insert(index, object)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/location.py", line 231, in insert2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     location)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/location.py", line 135, in _check_image_location
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     store_api.check_location_metadata(location['metadata'])
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 375, in check_location_metadata
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     check_location_metadata(val[key], key=key)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 386, in check_location_metadata
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     % dict(key=key, type=type(val)))
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi BackendException: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.

** Affects: glance
     Importance: Undecided
     Assignee: Alan Bishop (alan-bishop)
         Status: New

** Changed in: glance
     Assignee: (unassigned) => Alan Bishop (alan-bishop)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1861501

Title:
  Store ID fetched from URI is incorrectly encoded under py27

Status in Glance:
  New

Bug description:
  Whenever a client updates the locations for an image, glance checks to
  be sure the locations metadata includes the backend store ID that it
  parses from the request URI. However, under py27 the parsed store ID
  will not be properly encoded (it will be a str, but needs to be
  unicode). This leads to a subsequent failure, when glance checks to
  verify the value isinstance(val, six.text_type) at [1].

  The problem can be triggered by having nova create an image from a
  running VM. Under these circumstances, the store ID parsed by [2] and
  added at [3] will be encoded incorrectly.

  [1] https://opendev.org/openstack/glance_store/src/branch/master/glance_store/backend.py#L381
  [2] https://opendev.org/openstack/glance/src/branch/master/glance/common/store_utils.py#L159
  [3] https://opendev.org/openstack/glance/src/branch/master/glance/common/store_utils.py#L195

  This leads to the following error:

  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi [req-81008491-28d6-442c-b95e-9b0d2f108213 11d8e44ae836459cbc32cd31d452f068 14efbf135f20447f8f45815b8dcb5249 - default default] Caught error: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.: BackendException: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi Traceback (most recent call last):
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1474, in __call__
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     request, **action_args)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1517, in dispatch
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     return method(*args, **kwargs)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 417, in wrapped
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     return func(self, req, *args, **kwargs)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 256, in update
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     change_method(req, image, change)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 309, in _do_add
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     self._do_add_locations(image, path[1], value)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 565, in _do_add_locations
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     image.locations.insert(pos, value)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 297, in _checker
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     return method(*args, **kwargs)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 292, in insert
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     return self.locations.insert(index, object)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/location.py", line 231, in insert2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     location)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/location.py", line 135, in _check_image_location
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     store_api.check_location_metadata(location['metadata'])
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 375, in check_location_metadata
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     check_location_metadata(val[key], key=key)
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 386, in check_location_metadata
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi     % dict(key=key, type=type(val)))
  2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi BackendException: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1861501/+subscriptions
Follow ups

[Bug 1861501] Re: Store ID fetched from URI is incorrectly encoded under py27
From: OpenStack Infra, 2020-01-31