yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1238604] Re: Run into 500 error during delete image

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Stuart McLaren <1238604@xxxxxxxxxxxxxxxxxx>
Date: Tue, 03 Dec 2013 11:26:56 -0000
Reply-to: Bug 1238604 <1238604@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
If you enable delayed delete you get an E500 and the following stack
trace:

Dec  2 11:29:17 gl-aw1rdc1-registry0000 4101 DEBUG eventlet.wsgi.server [f46a69d7-c49b-4e41-ad12-447bcd7d2c38 77049353665607 34096082065107] Traceback (most recent call last):
012  File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 384, in handle_one_response
012    result = self.application(self.environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/hp_glance_extras/middleware/healthcheck.py", line 38, in __call__
012    return self.app(env, start_response)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
012    resp = self.call_func(req, *args, **self.kwargs)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
012    return self.func(req, *args, **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line 377, in __call__
012    response = req.get_response(self.application)
012  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
012    application, catch_exc_info=False)
012  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
012    app_iter = application(self.environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/hp/middleware/cs_auth_token.py", line 160, in __call__
012    return super(CsAuthProtocol, self).__call__(env, start_response)
012  File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 539, in __call__
012    return self.app(env, start_response)
012  File "/usr/lib/python2.7/dist-packages/hp/middleware/cs_authz.py", line 30, in __call__
012    return self.app(env, start_response)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
012    resp = self.call_func(req, *args, **self.kwargs)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
012    return self.func(req, *args, **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line 377, in __call__
012    response = req.get_response(self.application)
012  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
012    application, catch_exc_info=False)
012  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
012    app_iter = application(self.environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
012    resp = self.call_func(req, *args, **self.kwargs)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
012    return self.func(req, *args, **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line 377, in __call__
012    response = req.get_response(self.application)
012  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
012    application, catch_exc_info=False)
012  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
012    app_iter = application(self.environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
012    resp = self.call_func(req, *args, **self.kwargs)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
012    return self.func(req, *args, **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line 377, in __call__
012    response = req.get_response(self.application)
012  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
012    application, catch_exc_info=False)
012  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
012    app_iter = application(self.environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in __call__
012    return app(environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
012    return resp(environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
012    response = self.app(environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
012    return resp(environ, start_response)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
012    resp = self.call_func(req, *args, **self.kwargs)
012  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
012    return self.func(req, *args, **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line 609, in __call__
012    request, **action_args)
012  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line 628, in dispatch
012    return method(*args, **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/common/utils.py", line 422, in wrapped
012    return func(self, req, *args, **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/api/v1/images.py", line 931, in delete
012    CONF.delayed_delete)
012  File "/usr/lib/python2.7/dist-packages/glance/api/v1/upload_utils.py", line 45, in initiate_deletion
012    location, id)
012  File "/usr/lib/python2.7/dist-packages/glance/store/__init__.py", line 293, in schedule_delayed_delete_from_backend
012    file_queue.add_location(image_id, uri)
012  File "/usr/lib/python2.7/dist-packages/glance/store/scrubber.py", line 175, in add_location
012    image = self.registry.get_image(image_id)
012  File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/client.py", line 143, in get_image
012    res = self.do_request("GET", "/images/%s" % image_id)
012  File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/client.py", line 110, in do_request
012    **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 67, in wrapped
012    return func(self, *args, **kwargs)
012  File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 383, in do_request
012    headers=copy.deepcopy(headers))
012  File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 81, in wrapped
012    return func(self, method, url, body, headers)
012  File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 524, in _do_request
012    raise exception.NotAuthenticated(res.read())
012NotAuthenticated: Authentication required



** Changed in: glance
       Status: Invalid => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1238604

Title:
  Run into 500 error during delete image

Status in OpenStack Image Registry and Delivery Service (Glance):
  Confirmed

Bug description:
  Recreate steps:
  1. Enable delayed delete
      delayed_delete = True

  2. Create a new image by:
      glance image-create --name flwang_1 --container-format bare --disk-format qcow2 --is-public yes --location https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img

  3. Delete image by:
      glance image-delete flwang_1

  You will see an error like below, but the image has been deleted, in 'pending-delete' state.
  Request returned failure status.
  HTTPInternalServerError (HTTP 500): Unable to delete image 86d0a3df-d140-4d41-aaae-f1c538591d3d

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1238604/+subscriptions