openstack team mailing list archive

Thread
Date
Unable to download images from glance.

To: openstack@xxxxxxxxxxxxxxxxxxx
From: Antonio Messina <antonio.s.messina@xxxxxxxxx>
Date: Thu, 7 Feb 2013 20:10:07 +0100
Sender: arcimboldo@xxxxxxxxx
Hi all,

I've recently updated the password of the glance, swift and nova
services via keystone user-update-password, and now I am unable to run
instances because nova-compute is unable to download the image from
glance.

I know it sounds stupid, but I am pretty sure all the configuration
files are fine, and the password works for *some* of the
functionalities but not for glance image-download. I tried "swift -U
service:glance ... download glance image-id" and it worked. Als
"glance image-list" works.
When I try "glance -d download-image ...", however, I get:

(cloud)antonio@kenny:~$ glance -d image-download
d5b25046-fd08-4e30-950b-ab956c547f8d --file /tmp/antanicurl -i -X GET
-H 'X-Auth-Token: 819f0cf1786e4d80ba0a1f672bbe9713' -H 'Content-Type:
application/octet-stream' -H 'User-Agent: python-glanceclient'
http://myhostname:9292/v1/images/d5b25046-fd08-4e30-950b-ab956c547f8d

HTTP/1.1 500 Internal Server Error
date: Thu, 07 Feb 2013 19:02:13 GMT
content-length: 5424
content-type: text/plain
connection: close

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 382,
in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line
326, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086,
in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055,
in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/keystone/middleware/auth_token.py",
line 278, in __call__
    return self.app(env, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line
326, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086,
in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055,
in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in __call__
    return app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line
131, in __call__
    response = self.app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line
532, in __call__
    request, **action_args)
  File "/usr/lib/python2.7/dist-packages/glance/common/wsgi.py", line
549, in dispatch
    return method(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/glance/api/v1/images.py",
line 318, in show
    image_meta['location'])
  File "/usr/lib/python2.7/dist-packages/glance/api/v1/images.py",
line 294, in _get_from_store
    image_data, image_size = get_from_backend(context, where)
  File "/usr/lib/python2.7/dist-packages/glance/store/__init__.py",
line 221, in get_from_backend
    return store.get(loc)
  File "/usr/lib/python2.7/dist-packages/glance/store/swift.py", line
291, in get
    resp_chunk_size=self.CHUNKSIZE)
  File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line
1048, in get_object
    resp_chunk_size=resp_chunk_size)
  File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line
969, in _retry
    self.url, self.token = self.get_auth()
  File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line
957, in get_auth
    os_options=self.os_options)
  File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line
302, in get_auth
    key, kwargs['os_options'])
  File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line
244, in get_keystoneclient_2_0
    auth_url=auth_url)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/v2_0/client.py",
line 80, in __init__
    self.authenticate()
  File "/usr/lib/python2.7/dist-packages/keystoneclient/v2_0/client.py",
line 102, in authenticate
    return_raw=True)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/v2_0/tokens.py",
line 37, in authenticate
    return self._create('/tokens', params, "access", return_raw=return_raw)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line
82, in _create
    resp, body = self.api.post(url, body=body)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/client.py",
line 180, in post
    return self._cs_request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/client.py",
line 167, in _cs_request
    self.authenticate()
  File "/usr/lib/python2.7/dist-packages/keystoneclient/v2_0/client.py",
line 102, in authenticate
    return_raw=True)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/v2_0/tokens.py",
line 37, in authenticate
    return self._create('/tokens', params, "access", return_raw=return_raw)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line
82, in _create
    resp, body = self.api.post(url, body=body)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/client.py",
line 180, in post
    return self._cs_request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/client.py",
line 161, in _cs_request
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/client.py",
line 141, in request
    raise exceptions.from_response(resp, body)
Unauthorized: Invalid user / password (HTTP 401)


Request returned failure status.
HTTPInternalServerError (HTTP 500)

regardless of the user I am using with glance.

>From the log of glance and swift it seems that glance is not
contacting swift (the store backend). However, the glance log looks
fine:

==> /var/log/glance/api.log <==
2013-02-07 20:07:36 7490 DEBUG
glance.api.middleware.version_negotiation [-] Determining version of
request: GET //v1/images/d5b25046-fd08-4e30-950b-ab956c547f8d Accept:
process_request
/usr/lib/python2.7/dist-packages/glance/api/middleware/version_negotiation.py:45
2013-02-07 20:07:36 7490 DEBUG
glance.api.middleware.version_negotiation [-] Using url versioning
process_request
/usr/lib/python2.7/dist-packages/glance/api/middleware/version_negotiation.py:58
2013-02-07 20:07:36 7490 DEBUG
glance.api.middleware.version_negotiation [-] Matched version: v1
process_request
/usr/lib/python2.7/dist-packages/glance/api/middleware/version_negotiation.py:70
2013-02-07 20:07:36 7490 DEBUG
glance.api.middleware.version_negotiation [-] new uri
/v1/images/d5b25046-fd08-4e30-950b-ab956c547f8d process_request
/usr/lib/python2.7/dist-packages/glance/api/middleware/version_negotiation.py:71
2013-02-07 20:07:36 DEBUG glance.api.policy
[9f346e0c-2a0e-45a3-a281-e7a3c9e85802 6f86d41e1244471d86a02f79c4e03ef1
4bdc5d18c711438f8be0ec9b70272892] Loaded policy rules: {u'default':
[], u'manage_image_cache': [[u'role:admin']]} load_rules
/usr/lib/python2.7/dist-packages/glance/api/policy.py:63
2013-02-07 20:07:36 DEBUG glance.api.policy
[9f346e0c-2a0e-45a3-a281-e7a3c9e85802 6f86d41e1244471d86a02f79c4e03ef1
4bdc5d18c711438f8be0ec9b70272892] Loaded policy rules: {u'default':
[], u'manage_image_cache': [[u'role:admin']]} load_rules
/usr/lib/python2.7/dist-packages/glance/api/policy.py:63
2013-02-07 20:07:36 DEBUG glance.common.client
[9f346e0c-2a0e-45a3-a281-e7a3c9e85802 6f86d41e1244471d86a02f79c4e03ef1
4bdc5d18c711438f8be0ec9b70272892] Constructed URL:
http://0.0.0.0:9191/images/d5b25046-fd08-4e30-950b-ab956c547f8d
_construct_url /usr/lib/python2.7/dist-packages/glance/common/client.py:464

==> /var/log/glance/registry.log <==
2013-02-07 20:07:36 INFO glance.registry.api.v1.images
[3cd682ee-0739-4161-a611-1e7f428a1072 6f86d41e1244471d86a02f79c4e03ef1
4bdc5d18c711438f8be0ec9b70272892] Successfully retrieved image
d5b25046-fd08-4e30-950b-ab956c547f8d

==> /var/log/glance/api.log <==
2013-02-07 20:07:36 DEBUG glance.registry.client
[9f346e0c-2a0e-45a3-a281-e7a3c9e85802 6f86d41e1244471d86a02f79c4e03ef1
4bdc5d18c711438f8be0ec9b70272892] Registry request GET
/images/d5b25046-fd08-4e30-950b-ab956c547f8d HTTP 200 request id
req-3cd682ee-0739-4161-a611-1e7f428a1072 do_request
/usr/lib/python2.7/dist-packages/glance/registry/client.py:94
2013-02-07 20:07:36 DEBUG glance.store.swift
[9f346e0c-2a0e-45a3-a281-e7a3c9e85802 6f86d41e1244471d86a02f79c4e03ef1
4bdc5d18c711438f8be0ec9b70272892] Creating Swift connection with
(auth_address=http://myhostname:5000/v2.0/, user=service:glance,
snet=False, auth_version=2) _make_swift_connection
/usr/lib/python2.7/dist-packages/glance/store/swift.py:363

Anyone have an idea on how to debug this?

.a.

-- 
antonio.s.messina@xxxxxxxxx
GC3: Grid Computing Competence Center
http://www.gc3.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland
Follow ups

Re: Unable to download images from glance.
From: Iccha Sethi, 2013-02-07