yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #48259
[Bug 1558808] Re: openstack liberty: glance crash on v1 api called from nova
problem solved.
I change the registry_host on /etc/glance/glance-api.conf to the vip interface.
registry_host=172.16.90.10
thanks to nikhil and bauzas that help me to address the problem
** Changed in: glance
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1558808
Title:
openstack liberty: glance crash on v1 api called from nova
Status in Glance:
Invalid
Bug description:
running 'nova image-list' gives the following crash
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'glanceclient.exc.HTTPInternalServerError'> (HTTP 500) (Request-ID: req-d834d6f8-e2fa-4f00-ac53-97efff2f1da7)
same command called by glance with api v2 works well
glance image-list
+--------------------------------------+--------+
| ID | Name |
+--------------------------------------+--------+
| 7fbaa9c3-c09b-4ddd-aad5-7b6d51111695 | cirros |
+--------------------------------------+--------+
the problem seems involved in the glance v1 api
I'm running centos 7 with the following nova,glance packages:
python-glanceclient-1.1.0-1.el7.noarch
python-glance-store-0.9.1-1.el7.noarch
openstack-glance-11.0.1-2.el7.noarch
python-glance-11.0.1-2.el7.noarch
openstack-nova-common-12.0.1-1.el7.noarch
openstack-nova-api-12.0.1-1.el7.noarch
python-novaclient-2.30.1-1.el7.noarch
python-nova-12.0.1-1.el7.noarch
openstack-nova-cert-12.0.1-1.el7.noarch
openstack-nova-novncproxy-12.0.1-1.el7.noarch
openstack-nova-scheduler-12.0.1-1.el7.noarch
openstack-nova-conductor-12.0.1-1.el7.noarch
openstack-nova-console-12.0.1-1.el7.noarch
here follows nova logs, glance logs, and the http flow of the error
nova
-----
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions [req-c84a66d7-8801-417e-bbca-5bd46c11182b a96dc90182dd499ebaa3814573336608 1af03692813f48f49dd0130588c53909 - - -] Unexpected exception in API method
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions return f(*args, **kwargs)
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/images.py", line 145, in detail
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions **page_params)
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/image/api.py", line 68, in get_all
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions return session.detail(context, **kwargs)
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 284, in detail
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions for image in images:
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/glanceclient/v1/images.py", line 254, in list
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions for image in paginate(params, return_request_id):
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/glanceclient/v1/images.py", line 238, in paginate
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions images, resp = self._list(url, "images")
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/glanceclient/v1/images.py", line 63, in _list
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions resp, body = self.client.get(url)
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 280, in get
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions return self._request('GET', url, **kwargs)
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 272, in _request
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions resp, body_iter = self._handle_response(resp)
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 93, in _handle_response
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions raise exc.from_response(resp, resp.content)
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions HTTPInternalServerError: 500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
2016-03-17 16:16:41.509 6316 ERROR nova.api.openstack.extensions
glance
--------
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi [req-f63ca99a-8663-4874-8118-cc032b913af1 a96dc90182dd499ebaa3814573336608 1af03692813f48f49dd0130588c53909 - - -] Caught error: [Errno 111] ECONNREFUSED
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi Traceback (most recent call last):
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 879, in __call__
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi request, **action_args)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 907, in dispatch
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi return method(*args, **kwargs)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v1/images.py", line 367, in detail
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi images = registry.get_images_detail(req.context, **params)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/registry/client/v1/api.py", line 155, in get_images_detail
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi return c.get_images_detailed(**kwargs)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py", line 150, in get_images_detailed
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi res = self.do_request("GET", "/images/detail", params=params)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py", line 136, in do_request
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi 'exc_name': exc_name})
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py", line 121, in do_request
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi **kwargs)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 71, in wrapped
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi return func(self, *args, **kwargs)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 375, in do_request
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi headers=copy.deepcopy(headers))
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 88, in wrapped
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi return func(self, method, url, body, headers)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 540, in _do_request
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi raise exception.ClientConnectionError(e)
2016-03-17 16:13:26.720 10767 ERROR glance.common.wsgi ClientConnectionError: [Errno 111] ECONNREFUSED
glance http v1 stream
-------------------------
GET /v1/images/detail?is_public=none&limit=20 HTTP/1.1
Host: 172.16.90.10:9292
Accept-Encoding: gzip, deflate
X-Tenant-Id: 1af03692813f48f49dd0130588c53909
Accept: */*
User-Agent: python-glanceclient
Connection: keep-alive
X-User-Id: a96dc90182dd499ebaa3814573336608
X-Auth-Token: 2e4ae379eead44afad88af328102af8f
X-Identity-Status: Confirmed
Content-Type: application/octet-stream
X-Roles: admin
HTTP/1.1 500 Internal Server Error
Content-Length: 228
Content-Type: text/html; charset=UTF-8
X-Openstack-Request-Id: req-e9fa43b9-b4a5-4462-b3a4-956fcb56a02e
Date: Thu, 17 Mar 2016 16:00:48 GMT
Connection: keep-alive
<html>
<head>
<title>500 Internal Server Error</title>
</head>
<body>
<h1>500 Internal Server Error</h1>
The server has either erred or is incapable of performing the requested operation.<br /><br />
</body>
</html>
glance http v2 stream
--------------------------
GET /v2/images?limit=20&sort_key=name&sort_dir=asc HTTP/1.1
Host: 172.16.90.10:9292
Accept-Encoding: gzip, deflate
Accept: */*
X-Auth-Token: 48dc48e173d246ae89757890dfc56657
Connection: keep-alive
User-Agent: python-glanceclient
Content-Type: application/octet-stream
HTTP/1.1 200 OK
Content-Length: 689
Content-Type: application/json; charset=UTF-8
X-Openstack-Request-Id: req-4159a122-4fb0-4495-807e-83f598050409
Date: Thu, 17 Mar 2016 16:00:45 GMT
Connection: keep-alive
{"images": [{"status": "active", "name": "cirros", "tags": [], "container_format": "bare", "created_at": "2016-03-16T17:43:27Z", "size": 13287936, "disk_format": "qcow2", "updated_at": "2016-03-16T17:43:27Z", "visibility": "public", "self": "/v2/images/7fbaa9c3-c09b-4ddd-aad5-7b6d51111695", "min_disk": 0, "protected": false, "id": "7fbaa9c3-c09b-4ddd-aad5-7b6d51111695", "file": "/v2/images/7fbaa9c3-c09b-4ddd-aad5-7b6d51111695/file", "checksum": "ee1eca47dc88f4879d8a229cc70a07c6", "owner": "1af03692813f48f49dd0130588c53909", "virtual_size": null, "min_ram": 0, "schema": "/v2/schemas/image"}], "schema": "/v2/schemas/images", "first": "/v2/images?sort_key=name&sort_dir=asc&limit=20"}GET /v2/schemas/image HTTP/1.1
Host: 172.16.90.10:9292
Accept-Encoding: gzip, deflate
Accept: */*
X-Auth-Token: 48dc48e173d246ae89757890dfc56657
Connection: keep-alive
User-Agent: python-glanceclient
Content-Type: application/octet-stream
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 3902
X-Openstack-Request-Id: req-1459fc0b-d282-4b74-b671-83a7b6374351
Date: Thu, 17 Mar 2016 16:00:45 GMT
Connection: keep-alive
{"additionalProperties": {"type": "string"}, "name": "image", "links":
[{"href": "{self}", "rel": "self"}, {"href": "{file}", "rel":
"enclosure"}, {"href": "{schema}", "rel": "describedby"}],
"properties": {"status": {"enum": ["queued", "saving", "active",
"killed", "deleted", "pending_delete", "deactivated"], "type":
"string", "description": "Status of the image (READ-ONLY)"}, "tags":
{"items": {"type": "string", "maxLength": 255}, "type": "array",
"description": "List of strings related to the image"}, "kernel_id":
{"pattern": "^([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0
-9a-fA-F]){4}-([0-9a-fA-F]){12}$", "type": ["null", "string"],
"description": "ID of image stored in Glance that should be used as
the kernel when booting an AMI-style image.", "is_base": false},
"container_format": {"enum": [null, "ami", "ari", "aki", "bare",
"ovf", "ova"], "type": ["null", "string"], "description": "Format of
the container"}, "min_ram": {"type": "integer", "description": "Amount
of ram (in MB) required to boot image."}, "ramdisk_id": {"pattern":
"^([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-
fA-F]){4}-([0-9a-fA-F]){12}$", "type": ["null", "string"],
"description": "ID of image stored in Glance that should be used as
the ramdisk when booting an AMI-style image.", "is_base": false},
"locations": {"items": {"required": ["url", "metadata"], "type":
"object", "properties": {"url": {"type": "string", "maxLength": 255},
"metadata": {"type": "object"}}}, "type": "array", "description": "A
set of URLs to access the image file kept in external store"},
"visibility": {"enum": ["public", "private"], "type": "string",
"description": "Scope of image accessibility"}, "updated_at": {"type":
"string", "description": "Date and time of the last image modification
(READ-ONLY)"}, "owner": {"type": ["null", "string"], "description":
"Owner of the image", "maxLength": 255}, "file": {"type": "string",
"description": "(READ-ONLY)"}, "min_disk": {"type": "integer",
"description": "Amount of disk space (in GB) required to boot
image."}, "virtual_size": {"type": ["null", "integer"], "description":
"Virtual size of image in bytes (READ-ONLY)"}, "id": {"pattern": "^([0
-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0
-9a-fA-F]){12}$", "type": "string", "description": "An identifier for
the image"}, "size": {"type": ["null", "integer"], "description":
"Size of image file in bytes (READ-ONLY)"}, "instance_uuid": {"type":
"string", "description": "ID of instance used to create this image.",
"is_base": false}, "os_distro": {"type": "string", "description":
"Common name of operating system distribution as specified in
http://docs.openstack.org/trunk/openstack-compute/admin/content
/adding-images.html", "is_base": false}, "name": {"type": ["null",
"string"], "description": "Descriptive name for the image",
"maxLength": 255}, "checksum": {"type": ["null", "string"],
"description": "md5 hash of image contents. (READ-ONLY)", "maxLength":
32}, "created_at": {"type": "string", "description": "Date and time of
image registration (READ-ONLY)"}, "disk_format": {"enum": [null,
"ami", "ari", "aki", "vhd", "vmdk", "raw", "qcow2", "vdi", "iso"],
"type": ["null", "string"], "description": "Format of the disk"},
"os_version": {"type": "string", "description": "Operating system
version as specified by the distributor", "is_base": false},
"protected": {"type": "boolean", "description": "If true, image will
not be deletable."}, "architecture": {"type": "string", "description":
"Operating system architecture as specified in
http://docs.openstack.org/trunk/openstack-compute/admin/content
/adding-images.html", "is_base": false}, "direct_url": {"type":
"string", "description": "URL to access the image file kept in
external store (READ-ONLY)"}, "self": {"type": "string",
"description": "(READ-ONLY)"}, "schema": {"type": "string",
"description": "(READ-ONLY)"}}}
nova http error stream
--------------------------
GET /v2/ HTTP/1.1
Host: 172.16.90.10:8774
Connection: keep-alive
User-Agent: python-novaclient
Accept-Encoding: gzip, deflate
Accept: application/json
X-Auth-Token: 391a552d5d9540718e65f1ca09fc0e50
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 377
X-Compute-Request-Id: req-d60364c4-c1e5-4fbf-8b79-17e2be3452d3
Date: Thu, 17 Mar 2016 16:00:53 GMT
Connection: keep-alive
{"version": {"status": "SUPPORTED", "updated": "2011-01-21T11:33:21Z", "links": [{"href": "http://172.16.90.10:8774/v2/";, "rel": "self"}, {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": "describedby"}], "min_version": "", "version": "", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2"}], "id": "v2.0"}}GET /v2/1af03692813f48f49dd0130588c53909/images/detail HTTP/1.1
Host: 172.16.90.10:8774
Connection: keep-alive
User-Agent: python-novaclient
Accept-Encoding: gzip, deflate
Accept: application/json
X-Auth-Token: 391a552d5d9540718e65f1ca09fc0e50
HTTP/1.1 500 Internal Server Error
Content-Length: 214
Content-Type: application/json; charset=UTF-8
X-Compute-Request-Id: req-d5bbea30-d67b-40bf-845a-d5f1d5ff3949
Date: Thu, 17 Mar 2016 16:00:54 GMT
Connection: keep-alive
{"computeFault": {"message": "Unexpected API Error. Please report this
at http://bugs.launchpad.net/nova/ and attach the Nova API log if
possible.\n<class 'glanceclient.exc.HTTPInternalServerError'>",
"code": 500}}
To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1558808/+subscriptions
References
