yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1550086] [NEW] 500 error when image is in use though the backend store

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Fei Long Wang <flwang@xxxxxxxxxxxxxxx>
Date: Fri, 26 Feb 2016 01:33:18 -0000
Reply-to: Bug 1550086 <1550086@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

When delete image from Glance and if the image is still in use in RBD,
Glance will raise 500 error as below:

====================================================================================
2016-02-26 14:23:42.849 2190 ERROR glance.common.wsgi [req-e1ca13af-7662-4122-a3a0-f64e8d75e6c1 79f60e6ab1bf4b2fbe7987c6c2b57e63 94b566de52f9423fab80ceee8c0a4a23 - - -] Caught error: The image cannot be deleted because it is in use through the backend store outside of Glance.
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi Traceback (most recent call last):
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/wsgi.py", line 881, in __call__
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     request, **action_args)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/wsgi.py", line 909, in dispatch
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     return method(*args, **kwargs)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/utils.py", line 508, in wrapped
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     return func(self, req, *args, **kwargs)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/api/v1/images.py", line 1099, in delete
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     {'status': ori_status})
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/api/v1/images.py", line 1095, in delete
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     upload_utils.initiate_deletion(req, loc_data, id)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/api/v1/upload_utils.py", line 46, in initiate_deletion
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     id, location_data)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/store_utils.py", line 124, in delete_image_location_from_backend
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     safe_delete_from_backend(context, image_id, location)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/store_utils.py", line 58, in safe_delete_from_backend
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     ret = store_api.delete_from_backend(location['url'], context=context)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance_store/backend.py", line 290, in delete_from_backend
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     return store.delete(loc, context=context)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance_store/capabilities.py", line 226, in op_checker
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     return store_op_fun(store, *args, **kwargs)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 410, in delete
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     self._delete_image(target_pool, loc.image, loc.snapshot)
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 304, in _delete_image
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     raise exceptions.InUseByStore()
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi InUseByStore: The image cannot be deleted because it is in use through the backend store outside of Glance.
2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi 
2016-02-26 14:23:42.902 2190 INFO eventlet.wsgi.server [req-e1ca13af-7662-4122-a3a0-f64e8d75e6c1 79f60e6ab1bf4b2fbe7987c6c2b57e63 94b566de52f9423fab80ceee8c0a4a23 - - -] 10.13.0.41 - - [26/Feb/2016 14:23:42] "DELETE /v1/images/xxxxxxxxxxxxxxxxxxxxxxx HTTP/1.1" 500 430 0.402856

** Affects: glance
     Importance: Undecided
     Assignee: Fei Long Wang (flwang)
         Status: New

** Changed in: glance
     Assignee: (unassigned) => Fei Long Wang (flwang)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1550086

Title:
  500 error when image is in use though the backend store

Status in Glance:
  New

Bug description:
  When delete image from Glance and if the image is still in use in RBD,
  Glance will raise 500 error as below:

  ====================================================================================
  2016-02-26 14:23:42.849 2190 ERROR glance.common.wsgi [req-e1ca13af-7662-4122-a3a0-f64e8d75e6c1 79f60e6ab1bf4b2fbe7987c6c2b57e63 94b566de52f9423fab80ceee8c0a4a23 - - -] Caught error: The image cannot be deleted because it is in use through the backend store outside of Glance.
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi Traceback (most recent call last):
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/wsgi.py", line 881, in __call__
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     request, **action_args)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/wsgi.py", line 909, in dispatch
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     return method(*args, **kwargs)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/utils.py", line 508, in wrapped
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     return func(self, req, *args, **kwargs)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/api/v1/images.py", line 1099, in delete
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     {'status': ori_status})
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/api/v1/images.py", line 1095, in delete
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     upload_utils.initiate_deletion(req, loc_data, id)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/api/v1/upload_utils.py", line 46, in initiate_deletion
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     id, location_data)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/store_utils.py", line 124, in delete_image_location_from_backend
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     safe_delete_from_backend(context, image_id, location)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance/common/store_utils.py", line 58, in safe_delete_from_backend
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     ret = store_api.delete_from_backend(location['url'], context=context)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance_store/backend.py", line 290, in delete_from_backend
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     return store.delete(loc, context=context)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance_store/capabilities.py", line 226, in op_checker
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     return store_op_fun(store, *args, **kwargs)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 410, in delete
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     self._delete_image(target_pool, loc.image, loc.snapshot)
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi   File "/opt/cat/openstack/glance/local/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 304, in _delete_image
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi     raise exceptions.InUseByStore()
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi InUseByStore: The image cannot be deleted because it is in use through the backend store outside of Glance.
  2016-02-26 14:23:42.849 2190 TRACE glance.common.wsgi 
  2016-02-26 14:23:42.902 2190 INFO eventlet.wsgi.server [req-e1ca13af-7662-4122-a3a0-f64e8d75e6c1 79f60e6ab1bf4b2fbe7987c6c2b57e63 94b566de52f9423fab80ceee8c0a4a23 - - -] 10.13.0.41 - - [26/Feb/2016 14:23:42] "DELETE /v1/images/xxxxxxxxxxxxxxxxxxxxxxx HTTP/1.1" 500 430 0.402856

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1550086/+subscriptions