yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #33675
[Bug 1464034] Re: Kilo glance image create stuck in saving and queued state
Further investigation found that the upload failure was due to
http_proxy. It prevented glanceclient from accessing API port which
happen to be on a public IP. Using private IPs might mask this issue.
** Changed in: glance
Assignee: (unassigned) => Alfred Shen (alfredcs)
** Changed in: glance
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1464034
Title:
Kilo glance image create stuck in saving and queued state
Status in OpenStack Image Registry and Delivery Service (Glance):
Invalid
Bug description:
In Kilo release, glance image-create with or without --os-image-api-
version caused "queued" status with following debug output. No error
log on keystone-api/registry.log. Othe glance CLIs work OK.
Similar symptom was reported in
https://bugs.launchpad.net/bugs/1146830 but seemed to be with diff
cause.
# dpkg -l | grep glance
ii glance 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Image Registry and Delivery Service - Daemons
ii glance-api 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Image Registry and Delivery Service - API
ii glance-common 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Image Registry and Delivery Service - Common
ii glance-registry 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Image Registry and Delivery Service - Registry
ii python-glance 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Image Registry and Delivery Service - Python library
ii python-glance-store 0.4.0-0ubuntu1~cloud0 all OpenStack Image Service store library - Python 2.x
ii python-glanceclient 1:0.15.0-0ubuntu1~cloud0 all Client library for Openstack glance server.
$ glance --debug --os-image-api-version 2 image-create --file /tmp/cirros-0.3.4-x86_64-disk.img --disk-format qcow2 --container-format bare --progress
curl -i -X GET -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X-Auth-Token: {SHA1}252d465682ed3a3d092b0ce05954601afb56c7df' -H 'Content-Type: application/octet-stream' http://3.39.89.230:9292/v2/schemas/image
HTTP/1.0 200 OK
content-length: 3867
via: 1.0 sjc1intproxy01 (squid/3.1.10)
x-cache: MISS from sjc1intproxy01
x-cache-lookup: MISS from sjc1intproxy01:8080
connection: keep-alive
date: Wed, 10 Jun 2015 21:41:24 GMT
content-type: application/json; charset=UTF-8
x-openstack-request-id: req-req-d0430e7a-5e36-466d-8d79-afefe9737695
{"additionalProperties": {"type": "string"}, "name": "image", "links":
[{"href": "{self}", "rel": "self"}, {"href": "{file}", "rel":
"enclosure"}, {"href": "{schema}", "rel": "describedby"}],
"properties": {"status": {"enum": ["queued", "saving", "active",
"killed", "deleted", "pending_delete"], "type": "string",
"description": "Status of the image (READ-ONLY)"}, "tags": {"items":
{"type": "string", "maxLength": 255}, "type": "array", "description":
"List of strings related to the image"}, "kernel_id": {"pattern":
"^([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-
fA-F]){4}-([0-9a-fA-F]){12}$", "type": "string", "description": "ID of
image stored in Glance that should be used as the kernel when booting
an AMI-style image.", "is_base": false}, "container_format": {"enum":
[null, "ami", "ari", "aki", "bare", "ovf", "ova"], "type": ["null",
"string"], "description": "Format of the container"}, "min_ram":
{"type": "integer", "description": "Amount of ram (in MB) required to
boot image."}, "ramdisk_id": {"pattern": "^([0-9a-fA-F]){8}-([0-9a-
fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){12}$",
"type": "string", "description": "ID of image stored in Glance that
should be used as the ramdisk when booting an AMI-style image.",
"is_base": false}, "locations": {"items": {"required": ["url",
"metadata"], "type": "object", "properties": {"url": {"type":
"string", "maxLength": 255}, "metadata": {"type": "object"}}}, "type":
"array", "description": "A set of URLs to access the image file kept
in external store"}, "visibility": {"enum": ["public", "private"],
"type": "string", "description": "Scope of image accessibility"},
"updated_at": {"type": "string", "description": "Date and time of the
last image modification (READ-ONLY)"}, "owner": {"type": ["null",
"string"], "description": "Owner of the image", "maxLength": 255},
"file": {"type": "string", "description": "(READ-ONLY)"}, "min_disk":
{"type": "integer", "description": "Amount of disk space (in GB)
required to boot image."}, "virtual_size": {"type": ["null",
"integer"], "description": "Virtual size of image in bytes (READ-
ONLY)"}, "id": {"pattern": "^([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-
fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){12}$", "type": "string",
"description": "An identifier for the image"}, "size": {"type":
["null", "integer"], "description": "Size of image file in bytes
(READ-ONLY)"}, "instance_uuid": {"type": "string", "description": "ID
of instance used to create this image.", "is_base": false},
"os_distro": {"type": "string", "description": "Common name of
operating system distribution as specified in
http://docs.openstack.org/trunk/openstack-compute/admin/content
/adding-images.html", "is_base": false}, "name": {"type": ["null",
"string"], "description": "Descriptive name for the image",
"maxLength": 255}, "checksum": {"type": ["null", "string"],
"description": "md5 hash of image contents. (READ-ONLY)", "maxLength":
32}, "created_at": {"type": "string", "description": "Date and time of
image registration (READ-ONLY)"}, "disk_format": {"enum": [null,
"ami", "ari", "aki", "vhd", "vmdk", "raw", "qcow2", "vdi", "iso"],
"type": ["null", "string"], "description": "Format of the disk"},
"os_version": {"type": "string", "description": "Operating system
version as specified by the distributor", "is_base": false},
"protected": {"type": "boolean", "description": "If true, image will
not be deletable."}, "architecture": {"type": "string", "description":
"Operating system architecture as specified in
http://docs.openstack.org/trunk/openstack-compute/admin/content
/adding-images.html", "is_base": false}, "direct_url": {"type":
"string", "description": "URL to access the image file kept in
external store (READ-ONLY)"}, "self": {"type": "string",
"description": "(READ-ONLY)"}, "schema": {"type": "string",
"description": "(READ-ONLY)"}}}
curl -i -X GET -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H
'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X
-Auth-Token: {SHA1}252d465682ed3a3d092b0ce05954601afb56c7df' -H
'Content-Type: application/octet-stream'
http://3.39.89.230:9292/v2/schemas/image
HTTP/1.0 200 OK
content-length: 3867
via: 1.0 sjc1intproxy01 (squid/3.1.10)
x-cache: MISS from sjc1intproxy01
x-cache-lookup: MISS from sjc1intproxy01:8080
connection: keep-alive
date: Wed, 10 Jun 2015 21:41:24 GMT
content-type: application/json; charset=UTF-8
x-openstack-request-id: req-req-535be2e3-66a4-4cd9-b88f-183a2acb30c0
{"additionalProperties": {"type": "string"}, "name": "image", "links":
[{"href": "{self}", "rel": "self"}, {"href": "{file}", "rel":
"enclosure"}, {"href": "{schema}", "rel": "describedby"}],
"properties": {"status": {"enum": ["queued", "saving", "active",
"killed", "deleted", "pending_delete"], "type": "string",
"description": "Status of the image (READ-ONLY)"}, "tags": {"items":
{"type": "string", "maxLength": 255}, "type": "array", "description":
"List of strings related to the image"}, "kernel_id": {"pattern":
"^([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-
fA-F]){4}-([0-9a-fA-F]){12}$", "type": "string", "description": "ID of
image stored in Glance that should be used as the kernel when booting
an AMI-style image.", "is_base": false}, "container_format": {"enum":
[null, "ami", "ari", "aki", "bare", "ovf", "ova"], "type": ["null",
"string"], "description": "Format of the container"}, "min_ram":
{"type": "integer", "description": "Amount of ram (in MB) required to
boot image."}, "ramdisk_id": {"pattern": "^([0-9a-fA-F]){8}-([0-9a-
fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){12}$",
"type": "string", "description": "ID of image stored in Glance that
should be used as the ramdisk when booting an AMI-style image.",
"is_base": false}, "locations": {"items": {"required": ["url",
"metadata"], "type": "object", "properties": {"url": {"type":
"string", "maxLength": 255}, "metadata": {"type": "object"}}}, "type":
"array", "description": "A set of URLs to access the image file kept
in external store"}, "visibility": {"enum": ["public", "private"],
"type": "string", "description": "Scope of image accessibility"},
"updated_at": {"type": "string", "description": "Date and time of the
last image modification (READ-ONLY)"}, "owner": {"type": ["null",
"string"], "description": "Owner of the image", "maxLength": 255},
"file": {"type": "string", "description": "(READ-ONLY)"}, "min_disk":
{"type": "integer", "description": "Amount of disk space (in GB)
required to boot image."}, "virtual_size": {"type": ["null",
"integer"], "description": "Virtual size of image in bytes (READ-
ONLY)"}, "id": {"pattern": "^([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-
fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){12}$", "type": "string",
"description": "An identifier for the image"}, "size": {"type":
["null", "integer"], "description": "Size of image file in bytes
(READ-ONLY)"}, "instance_uuid": {"type": "string", "description": "ID
of instance used to create this image.", "is_base": false},
"os_distro": {"type": "string", "description": "Common name of
operating system distribution as specified in
http://docs.openstack.org/trunk/openstack-compute/admin/content
/adding-images.html", "is_base": false}, "name": {"type": ["null",
"string"], "description": "Descriptive name for the image",
"maxLength": 255}, "checksum": {"type": ["null", "string"],
"description": "md5 hash of image contents. (READ-ONLY)", "maxLength":
32}, "created_at": {"type": "string", "description": "Date and time of
image registration (READ-ONLY)"}, "disk_format": {"enum": [null,
"ami", "ari", "aki", "vhd", "vmdk", "raw", "qcow2", "vdi", "iso"],
"type": ["null", "string"], "description": "Format of the disk"},
"os_version": {"type": "string", "description": "Operating system
version as specified by the distributor", "is_base": false},
"protected": {"type": "boolean", "description": "If true, image will
not be deletable."}, "architecture": {"type": "string", "description":
"Operating system architecture as specified in
http://docs.openstack.org/trunk/openstack-compute/admin/content
/adding-images.html", "is_base": false}, "direct_url": {"type":
"string", "description": "URL to access the image file kept in
external store (READ-ONLY)"}, "self": {"type": "string",
"description": "(READ-ONLY)"}, "schema": {"type": "string",
"description": "(READ-ONLY)"}}}
curl -i -X POST -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*'
-H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X
-Auth-Token: {SHA1}252d465682ed3a3d092b0ce05954601afb56c7df' -H
'Content-Type: application/json' -d '{"container_format": "bare",
"disk_format": "qcow2"}' http://3.39.89.230:9292/v2/images
HTTP/1.0 201 Created
content-length: 547
via: 1.0 sjc1intproxy01 (squid/3.1.10)
x-cache: MISS from sjc1intproxy01
x-cache-lookup: MISS from sjc1intproxy01:8080
connection: keep-alive
location: http://3.39.89.230:9292/v2/images/383b51bd-2ca1-429d-ac70-49187bc685c6
date: Wed, 10 Jun 2015 21:41:25 GMT
content-type: application/json; charset=UTF-8
x-openstack-request-id: req-req-76eae7da-a2fd-4037-aa5d-0c66e9b515b6
{"status": "queued", "name": null, "tags": [], "container_format":
"bare", "created_at": "2015-06-10T21:41:25Z", "size": null,
"disk_format": "qcow2", "updated_at": "2015-06-10T21:41:25Z",
"visibility": "private", "self": "/v2/images/383b51bd-2ca1-429d-
ac70-49187bc685c6", "min_disk": 0, "protected": false, "id":
"383b51bd-2ca1-429d-ac70-49187bc685c6", "file": "/v2/images/383b51bd-
2ca1-429d-ac70-49187bc685c6/file", "checksum": null, "owner":
"a3b62a7ed7154ea5bfe189c90b553690", "virtual_size": null, "min_ram":
0, "schema": "/v2/schemas/image"}
curl -i -X PUT -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X-Auth-Token: {SHA1}252d465682ed3a3d092b0ce05954601afb56c7df' -H 'Content-Type: application/octet-stream' http://3.39.89.230:9292/v2/images/383b51bd-2ca1-429d-ac70-49187bc685c6/file
[==> ] 5%+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | None |
| container_format | bare |
| created_at | 2015-06-10T21:41:25Z |
| disk_format | qcow2 |
| id | 383b51bd-2ca1-429d-ac70-49187bc685c6 |
| min_disk | 0 |
| min_ram | 0 |
| name | None |
| owner | a3b62a7ed7154ea5bfe189c90b553690 |
| protected | False |
| size | None |
| status | queued |
| tags | [] |
| updated_at | 2015-06-10T21:41:25Z |
| virtual_size | None |
| visibility | private |
+------------------+--------------------------------------+
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/glanceclient/shell.py", line 637, in main
args.func(client, args)
File "/usr/lib/python2.7/dist-packages/glanceclient/v2/shell.py", line 73, in do_image_create
do_image_upload(gc, args)
File "/usr/lib/python2.7/dist-packages/glanceclient/v2/shell.py", line 258, in do_image_upload
gc.images.upload(args.id, image_data, args.size)
File "/usr/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 138, in upload
self.http_client.put(url, headers=hdrs, data=body)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 266, in put
return self._request('PUT', url, **kwargs)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 215, in _request
raise exc.CommunicationError(message=message)
CommunicationError: Error finding address for http://3.39.89.230:9292/v2/images/383b51bd-2ca1-429d-ac70-49187bc685c6/file: [Errno 32] Broken pipe
Error finding address for http://3.39.89.230:9292/v2/images/383b51bd-2ca1-429d-ac70-49187bc685c6/file: [Errno 32] Broken pipe
To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1464034/+subscriptions
References
