← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 2006473] [NEW] ImageLocationsTest.test_replace_location fails with The HTTP URL is invalid: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

 

Public bug reported:

t1.3: tempest.api.image.v2.test_images.ImageLocationsTest.test_replace_location[id-bf6e0009-c039-4884-b498-db074caadb10]testtools.testresult.real._StringException: pythonlogging:'': {{{
2023-02-01 11:49:08,439 114634 INFO     [tempest.lib.common.rest_client] Request (ImageLocationsTest:test_replace_location): 201 POST https://149.202.176.133/image/v2/images 0.686s
2023-02-01 11:49:08,440 114634 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"container_format": "bare", "disk_format": "raw"}
    Response - Headers: {'date': 'Wed, 01 Feb 2023 11:49:08 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '626', 'content-type': 'application/json', 'location': 'http://149.202.176.133:19292/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496', 'openstack-image-import-methods': 'glance-direct,web-download,copy-image', 'openstack-image-store-ids': 'cheap,robust,web,os_glance_staging_store,os_glance_tasks_store', 'x-openstack-request-id': 'req-8ba71980-856b-4a34-b1b6-5957bf1feb2a', 'connection': 'close', 'status': '201', 'content-location': 'https://149.202.176.133/image/v2/images'}
        Body: b'{"name": null, "disk_format": "raw", "container_format": "bare", "visibility": "shared", "size": null, "virtual_size": null, "status": "queued", "checksum": null, "protected": false, "min_ram": 0, "min_disk": 0, "owner": "7ecb0d9a73b9472f962c12df325cccb2", "os_hidden": false, "os_hash_algo": null, "os_hash_value": null, "id": "ee8671ea-4771-461b-b14b-f4d5b4598496", "created_at": "2023-02-01T11:49:08Z", "updated_at": "2023-02-01T11:49:08Z", "locations": [], "tags": [], "self": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496", "file": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496/file", "schema": "/v2/schemas/image"}'
2023-02-01 11:49:09,845 114634 INFO     [tempest.lib.common.rest_client] Request (ImageLocationsTest:test_replace_location): 200 PATCH https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496 1.405s
2023-02-01 11:49:09,846 114634 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/openstack-images-v2.0-json-patch', 'X-Auth-Token': '<omitted>'}
        Body: [{"add": "/locations/-", "value": {"metadata": {"foo": "bar"}, "url": "http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz"}}]
    Response - Headers: {'date': 'Wed, 01 Feb 2023 11:49:09 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '770', 'content-type': 'application/json', 'x-openstack-request-id': 'req-834e7836-8ce7-4945-9a78-67f7f9f6460c', 'connection': 'close', 'status': '200', 'content-location': 'https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496'}
        Body: b'{"name": null, "disk_format": "raw", "container_format": "bare", "visibility": "shared", "size": 8633554, "virtual_size": null, "status": "active", "checksum": null, "protected": false, "min_ram": 0, "min_disk": 0, "owner": "7ecb0d9a73b9472f962c12df325cccb2", "os_hidden": false, "os_hash_algo": null, "os_hash_value": null, "id": "ee8671ea-4771-461b-b14b-f4d5b4598496", "created_at": "2023-02-01T11:49:08Z", "updated_at": "2023-02-01T11:49:10Z", "locations": [{"metadata": {"foo": "bar", "store": "web"}, "url": "http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz"}], "tags": [], "self": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496", "file": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496/file", "schema": "/v2/schemas/image", "stores": "web"}'
2023-02-01 11:49:09,880 114634 INFO     [tempest.lib.common.rest_client] Request (ImageLocationsTest:test_replace_location): 200 GET https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496 0.034s
2023-02-01 11:49:09,880 114634 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Wed, 01 Feb 2023 11:49:09 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '770', 'content-type': 'application/json', 'x-openstack-request-id': 'req-38f7d464-db27-4e09-be07-4e1f60e3bb47', 'connection': 'close', 'status': '200', 'content-location': 'https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496'}
        Body: b'{"name": null, "disk_format": "raw", "container_format": "bare", "visibility": "shared", "size": 8633554, "virtual_size": null, "status": "active", "checksum": null, "protected": false, "min_ram": 0, "min_disk": 0, "owner": "7ecb0d9a73b9472f962c12df325cccb2", "os_hidden": false, "os_hash_algo": null, "os_hash_value": null, "id": "ee8671ea-4771-461b-b14b-f4d5b4598496", "created_at": "2023-02-01T11:49:08Z", "updated_at": "2023-02-01T11:49:10Z", "locations": [{"url": "http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz";, "metadata": {"foo": "bar", "store": "web"}}], "tags": [], "self": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496", "file": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496/file", "schema": "/v2/schemas/image", "stores": "web"}'
2023-02-01 11:49:10,100 114634 INFO     [tempest.lib.common.rest_client] Request (ImageLocationsTest:test_replace_location): 400 PATCH https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496 0.219s
2023-02-01 11:49:10,100 114634 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/openstack-images-v2.0-json-patch', 'X-Auth-Token': '<omitted>'}
        Body: [{"add": "/locations/-", "value": {"metadata": {"speed": "88mph"}, "url": "http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz#new"}}]
    Response - Headers: {'date': 'Wed, 01 Feb 2023 11:49:10 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '38', 'content-type': 'text/plain; charset=UTF-8', 'x-openstack-request-id': 'req-3bd97e63-562a-403e-8ca0-7e0da0f38c05', 'connection': 'close', 'status': '400', 'content-location': 'https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496'}
        Body: b'400 Bad Request\n\nInvalid location\n\n   '
}}}

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/api/image/v2/test_images.py", line 875, in test_replace_location
    image = self._check_set_multiple_locations()
  File "/opt/stack/tempest/tempest/api/image/v2/test_images.py", line 851, in _check_set_multiple_locations
    self.client.update_image(image['id'], [
  File "/opt/stack/tempest/tempest/lib/services/image/v2/images_client.py", line 40, in update_image
    resp, body = self.patch('images/%s' % image_id, data, headers)
  File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 346, in patch
    return self.request('PATCH', url, extra_headers, headers, body)
  File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 720, in request
    self._error_checker(resp, resp_body)
  File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 831, in _error_checker
    raise exceptions.BadRequest(resp_body, resp=resp)
tempest.lib.exceptions.BadRequest: Bad request
Details: b'400 Bad Request\n\nInvalid location\n\n   '

In the glance log:

Feb 01 11:49:10.096936 np0032932858 glance-api[109935]: DEBUG
oslo_policy.policy [None req-afaf805e-e0bb-408f-9151-5a4c9d5ad5e1
tempest-ServerActionsTestJSON-1947886094 tempest-
ServerActionsTestJSON-1947886094-project] enforce: rule="get_image"
creds={"domain_id": null, "is_admin_project": true, "project_domain_id":
"default", "project_id": "d198379033af4d02aaa63c069e661fef", "roles":
["reader", "member"], "service_project_domain_id": "default",
"service_project_id": "ad0ce459e9f9448e9632cd0df5cd5ccd",
"service_roles": ["reader", "service", "admin", "member"],
"service_user_domain_id": "default", "service_user_id":
"0814687eb89f4de594732d1dc83e69ed", "system_scope": null,
"user_domain_id": "default", "user_id":
"2049143f1a2e4d39b38bed227226db61"} target={"backup_type": "daily",
"base_image_ref": "22617963-7650-423c-8a9f-7572da6026e2", "boot_roles":
"reader,member", "checksum": null, "container_format": "bare",
"created_at": "2023-02-01T11:48:58.000000", "disk_format": "raw",
"extra_properties": {}, "hw_cdrom_bus": "ide", "hw_disk_bus": "virtio",
"hw_machine_type": "pc", "hw_video_model": "virtio", "hw_vif_model":
"virtio", "image_id": "c0b70aa6-a445-48b4-8fd0-8f0b4b43fa52",
"image_location": "snapshot", "image_state": "available", "image_type":
"backup", "instance_uuid": "de308b29-2edd-4aee-886d-230ea41bd528",
"member": null, "min_disk": 1, "min_ram": 0, "name": "tempest-
backup-2-405031231", "os_hash_algo": null, "os_hash_value": null,
"os_hidden": false, "owner": "d198379033af4d02aaa63c069e661fef",
"owner_id": "d198379033af4d02aaa63c069e661fef", "owner_project_name":
"tempest-ServerActionsTestJSON-1947886094",
"owner_specified.openstack.md5": "", "owner_specified.openstack.object":
"images/cirros-raw", "owner_specified.openstack.sha256": "",
"owner_user_name": "tempest-ServerActionsTestJSON-1947886094-project",
"project_id": "d198379033af4d02aaa63c069e661fef", "protected": false,
"size": 1073741824, "status": "active", "tags": [], "updated_at":
"2023-02-01T11:49:03.000000", "user_id":
"2049143f1a2e4d39b38bed227226db61", "virtual_size": null, "visibility":
"private"} {{(pid=109935) enforce /usr/local/lib/python3.8/dist-
packages/oslo_policy/policy.py:1036}}

Feb 01 11:49:10.097319 np0032932858 glance-api[109934]: INFO
glance_store._drivers.http [None
req-3bd97e63-562a-403e-8ca0-7e0da0f38c05 tempest-
ImageLocationsTest-1995906154 tempest-
ImageLocationsTest-1995906154-project] The HTTP URL is invalid:
('Connection aborted.', RemoteDisconnected('Remote end closed connection
without response'))

Feb 01 11:49:10.098669 np0032932858 glance-api[109935]: DEBUG
oslo_policy.policy [None req-afaf805e-e0bb-408f-9151-5a4c9d5ad5e1
tempest-ServerActionsTestJSON-1947886094 tempest-
ServerActionsTestJSON-1947886094-project] enforce: rule="get_image"
creds={"domain_id": null, "is_admin_project": true, "project_domain_id":
"default", "project_id": "d198379033af4d02aaa63c069e661fef", "roles":
["reader", "member"], "service_project_domain_id": "default",
"service_project_id": "ad0ce459e9f9448e9632cd0df5cd5ccd",
"service_roles": ["reader", "service", "admin", "member"],
"service_user_domain_id": "default", "service_user_id":
"0814687eb89f4de594732d1dc83e69ed", "system_scope": null,
"user_domain_id": "default", "user_id":
"2049143f1a2e4d39b38bed227226db61"} target={"backup_type": "daily",
"base_image_ref": "22617963-7650-423c-8a9f-7572da6026e2", "boot_roles":
"reader,member", "checksum": null, "container_format": "bare",
"created_at": "2023-02-01T11:48:51.000000", "disk_format": "raw",
"extra_properties": {}, "hw_cdrom_bus": "ide", "hw_disk_bus": "virtio",
"hw_machine_type": "pc", "hw_video_model": "virtio", "hw_vif_model":
"virtio", "image_id": "db4a3513-7ee7-43db-a184-44b482eb0fc2",
"image_location": "snapshot", "image_state": "available", "image_type":
"backup", "instance_uuid": "de308b29-2edd-4aee-886d-230ea41bd528",
"member": null, "min_disk": 1, "min_ram": 0, "name": "tempest-
backup-1-916792853", "os_hash_algo": null, "os_hash_value": null,
"os_hidden": false, "owner": "d198379033af4d02aaa63c069e661fef",
"owner_id": "d198379033af4d02aaa63c069e661fef", "owner_project_name":
"tempest-ServerActionsTestJSON-1947886094",
"owner_specified.openstack.md5": "", "owner_specified.openstack.object":
"images/cirros-raw", "owner_specified.openstack.sha256": "",
"owner_user_name": "tempest-ServerActionsTestJSON-1947886094-project",
"project_id": "d198379033af4d02aaa63c069e661fef", "protected": false,
"size": 1073741824, "status": "active", "tags": [], "updated_at":
"2023-02-01T11:48:57.000000", "user_id":
"2049143f1a2e4d39b38bed227226db61", "virtual_size": null, "visibility":
"private"} {{(pid=109935) enforce /usr/local/lib/python3.8/dist-
packages/oslo_policy/policy.py:1036}}

Feb 01 11:49:10.099258 np0032932858 glance-api[109934]: INFO
eventlet.wsgi.server [None req-3bd97e63-562a-403e-8ca0-7e0da0f38c05
tempest-ImageLocationsTest-1995906154 tempest-
ImageLocationsTest-1995906154-project]
149.202.176.133,149.202.176.133,149.202.176.133 - - [01/Feb/2023
11:49:10] "PATCH /v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496
HTTP/1.1" 400 254 0.211937

Example failed run:
https://zuul.opendev.org/t/openstack/build/e57489f5e9484160b005c329b205f932/logs


logsearch query (https://github.com/gibizer/zuul-log-search): 
$ logsearch log --job nova-ceph-multistore --result FAILURE --file  controller/logs/screen-g-api.txt 'tempest-ImageLocationsTest.*The HTTP URL is invalid: .*RemoteDisconnected.*Remote end closed connection without response' --days 10
[snip]:
Builds with matching logs 5/25:
+----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+
| uuid                             | finished            | project        | pipeline | review                            | branch |
+----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+
| a9b9bf62802b4c9b9421b138c5c56024 | 2023-02-07T11:01:21 | openstack/nova | check    | https://review.opendev.org/872432 | master |
| bc7699718de14023b3142c78a6e9f10c | 2023-02-02T22:24:16 | openstack/nova | check    | https://review.opendev.org/868236 | master |
| a75c88f94d614eeea6bfa2fadada1638 | 2023-02-02T13:03:24 | openstack/nova | check    | https://review.opendev.org/854823 | master |
| dc19fe23fd9849708b884915fcdfb086 | 2023-02-01T21:04:59 | openstack/nova | check    | https://review.opendev.org/872441 | master |
| e57489f5e9484160b005c329b205f932 | 2023-02-01T12:15:19 | openstack/nova | gate     | https://review.opendev.org/872204 | master |
+----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+

** Affects: glance
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/2006473

Title:
  ImageLocationsTest.test_replace_location fails with The HTTP URL is
  invalid: ('Connection aborted.', RemoteDisconnected('Remote end closed
  connection without response'))

Status in Glance:
  New

Bug description:
  t1.3: tempest.api.image.v2.test_images.ImageLocationsTest.test_replace_location[id-bf6e0009-c039-4884-b498-db074caadb10]testtools.testresult.real._StringException: pythonlogging:'': {{{
  2023-02-01 11:49:08,439 114634 INFO     [tempest.lib.common.rest_client] Request (ImageLocationsTest:test_replace_location): 201 POST https://149.202.176.133/image/v2/images 0.686s
  2023-02-01 11:49:08,440 114634 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
          Body: {"container_format": "bare", "disk_format": "raw"}
      Response - Headers: {'date': 'Wed, 01 Feb 2023 11:49:08 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '626', 'content-type': 'application/json', 'location': 'http://149.202.176.133:19292/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496', 'openstack-image-import-methods': 'glance-direct,web-download,copy-image', 'openstack-image-store-ids': 'cheap,robust,web,os_glance_staging_store,os_glance_tasks_store', 'x-openstack-request-id': 'req-8ba71980-856b-4a34-b1b6-5957bf1feb2a', 'connection': 'close', 'status': '201', 'content-location': 'https://149.202.176.133/image/v2/images'}
          Body: b'{"name": null, "disk_format": "raw", "container_format": "bare", "visibility": "shared", "size": null, "virtual_size": null, "status": "queued", "checksum": null, "protected": false, "min_ram": 0, "min_disk": 0, "owner": "7ecb0d9a73b9472f962c12df325cccb2", "os_hidden": false, "os_hash_algo": null, "os_hash_value": null, "id": "ee8671ea-4771-461b-b14b-f4d5b4598496", "created_at": "2023-02-01T11:49:08Z", "updated_at": "2023-02-01T11:49:08Z", "locations": [], "tags": [], "self": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496", "file": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496/file", "schema": "/v2/schemas/image"}'
  2023-02-01 11:49:09,845 114634 INFO     [tempest.lib.common.rest_client] Request (ImageLocationsTest:test_replace_location): 200 PATCH https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496 1.405s
  2023-02-01 11:49:09,846 114634 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/openstack-images-v2.0-json-patch', 'X-Auth-Token': '<omitted>'}
          Body: [{"add": "/locations/-", "value": {"metadata": {"foo": "bar"}, "url": "http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz"}}]
      Response - Headers: {'date': 'Wed, 01 Feb 2023 11:49:09 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '770', 'content-type': 'application/json', 'x-openstack-request-id': 'req-834e7836-8ce7-4945-9a78-67f7f9f6460c', 'connection': 'close', 'status': '200', 'content-location': 'https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496'}
          Body: b'{"name": null, "disk_format": "raw", "container_format": "bare", "visibility": "shared", "size": 8633554, "virtual_size": null, "status": "active", "checksum": null, "protected": false, "min_ram": 0, "min_disk": 0, "owner": "7ecb0d9a73b9472f962c12df325cccb2", "os_hidden": false, "os_hash_algo": null, "os_hash_value": null, "id": "ee8671ea-4771-461b-b14b-f4d5b4598496", "created_at": "2023-02-01T11:49:08Z", "updated_at": "2023-02-01T11:49:10Z", "locations": [{"metadata": {"foo": "bar", "store": "web"}, "url": "http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz"}], "tags": [], "self": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496", "file": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496/file", "schema": "/v2/schemas/image", "stores": "web"}'
  2023-02-01 11:49:09,880 114634 INFO     [tempest.lib.common.rest_client] Request (ImageLocationsTest:test_replace_location): 200 GET https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496 0.034s
  2023-02-01 11:49:09,880 114634 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
          Body: None
      Response - Headers: {'date': 'Wed, 01 Feb 2023 11:49:09 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '770', 'content-type': 'application/json', 'x-openstack-request-id': 'req-38f7d464-db27-4e09-be07-4e1f60e3bb47', 'connection': 'close', 'status': '200', 'content-location': 'https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496'}
          Body: b'{"name": null, "disk_format": "raw", "container_format": "bare", "visibility": "shared", "size": 8633554, "virtual_size": null, "status": "active", "checksum": null, "protected": false, "min_ram": 0, "min_disk": 0, "owner": "7ecb0d9a73b9472f962c12df325cccb2", "os_hidden": false, "os_hash_algo": null, "os_hash_value": null, "id": "ee8671ea-4771-461b-b14b-f4d5b4598496", "created_at": "2023-02-01T11:49:08Z", "updated_at": "2023-02-01T11:49:10Z", "locations": [{"url": "http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz";, "metadata": {"foo": "bar", "store": "web"}}], "tags": [], "self": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496", "file": "/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496/file", "schema": "/v2/schemas/image", "stores": "web"}'
  2023-02-01 11:49:10,100 114634 INFO     [tempest.lib.common.rest_client] Request (ImageLocationsTest:test_replace_location): 400 PATCH https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496 0.219s
  2023-02-01 11:49:10,100 114634 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/openstack-images-v2.0-json-patch', 'X-Auth-Token': '<omitted>'}
          Body: [{"add": "/locations/-", "value": {"metadata": {"speed": "88mph"}, "url": "http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz#new"}}]
      Response - Headers: {'date': 'Wed, 01 Feb 2023 11:49:10 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '38', 'content-type': 'text/plain; charset=UTF-8', 'x-openstack-request-id': 'req-3bd97e63-562a-403e-8ca0-7e0da0f38c05', 'connection': 'close', 'status': '400', 'content-location': 'https://149.202.176.133/image/v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496'}
          Body: b'400 Bad Request\n\nInvalid location\n\n   '
  }}}

  Traceback (most recent call last):
    File "/opt/stack/tempest/tempest/api/image/v2/test_images.py", line 875, in test_replace_location
      image = self._check_set_multiple_locations()
    File "/opt/stack/tempest/tempest/api/image/v2/test_images.py", line 851, in _check_set_multiple_locations
      self.client.update_image(image['id'], [
    File "/opt/stack/tempest/tempest/lib/services/image/v2/images_client.py", line 40, in update_image
      resp, body = self.patch('images/%s' % image_id, data, headers)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 346, in patch
      return self.request('PATCH', url, extra_headers, headers, body)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 720, in request
      self._error_checker(resp, resp_body)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 831, in _error_checker
      raise exceptions.BadRequest(resp_body, resp=resp)
  tempest.lib.exceptions.BadRequest: Bad request
  Details: b'400 Bad Request\n\nInvalid location\n\n   '

  In the glance log:

  Feb 01 11:49:10.096936 np0032932858 glance-api[109935]: DEBUG
  oslo_policy.policy [None req-afaf805e-e0bb-408f-9151-5a4c9d5ad5e1
  tempest-ServerActionsTestJSON-1947886094 tempest-
  ServerActionsTestJSON-1947886094-project] enforce: rule="get_image"
  creds={"domain_id": null, "is_admin_project": true,
  "project_domain_id": "default", "project_id":
  "d198379033af4d02aaa63c069e661fef", "roles": ["reader", "member"],
  "service_project_domain_id": "default", "service_project_id":
  "ad0ce459e9f9448e9632cd0df5cd5ccd", "service_roles": ["reader",
  "service", "admin", "member"], "service_user_domain_id": "default",
  "service_user_id": "0814687eb89f4de594732d1dc83e69ed", "system_scope":
  null, "user_domain_id": "default", "user_id":
  "2049143f1a2e4d39b38bed227226db61"} target={"backup_type": "daily",
  "base_image_ref": "22617963-7650-423c-8a9f-7572da6026e2",
  "boot_roles": "reader,member", "checksum": null, "container_format":
  "bare", "created_at": "2023-02-01T11:48:58.000000", "disk_format":
  "raw", "extra_properties": {}, "hw_cdrom_bus": "ide", "hw_disk_bus":
  "virtio", "hw_machine_type": "pc", "hw_video_model": "virtio",
  "hw_vif_model": "virtio", "image_id":
  "c0b70aa6-a445-48b4-8fd0-8f0b4b43fa52", "image_location": "snapshot",
  "image_state": "available", "image_type": "backup", "instance_uuid":
  "de308b29-2edd-4aee-886d-230ea41bd528", "member": null, "min_disk": 1,
  "min_ram": 0, "name": "tempest-backup-2-405031231", "os_hash_algo":
  null, "os_hash_value": null, "os_hidden": false, "owner":
  "d198379033af4d02aaa63c069e661fef", "owner_id":
  "d198379033af4d02aaa63c069e661fef", "owner_project_name": "tempest-
  ServerActionsTestJSON-1947886094", "owner_specified.openstack.md5":
  "", "owner_specified.openstack.object": "images/cirros-raw",
  "owner_specified.openstack.sha256": "", "owner_user_name": "tempest-
  ServerActionsTestJSON-1947886094-project", "project_id":
  "d198379033af4d02aaa63c069e661fef", "protected": false, "size":
  1073741824, "status": "active", "tags": [], "updated_at":
  "2023-02-01T11:49:03.000000", "user_id":
  "2049143f1a2e4d39b38bed227226db61", "virtual_size": null,
  "visibility": "private"} {{(pid=109935) enforce
  /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1036}}

  Feb 01 11:49:10.097319 np0032932858 glance-api[109934]: INFO
  glance_store._drivers.http [None
  req-3bd97e63-562a-403e-8ca0-7e0da0f38c05 tempest-
  ImageLocationsTest-1995906154 tempest-
  ImageLocationsTest-1995906154-project] The HTTP URL is invalid:
  ('Connection aborted.', RemoteDisconnected('Remote end closed
  connection without response'))

  Feb 01 11:49:10.098669 np0032932858 glance-api[109935]: DEBUG
  oslo_policy.policy [None req-afaf805e-e0bb-408f-9151-5a4c9d5ad5e1
  tempest-ServerActionsTestJSON-1947886094 tempest-
  ServerActionsTestJSON-1947886094-project] enforce: rule="get_image"
  creds={"domain_id": null, "is_admin_project": true,
  "project_domain_id": "default", "project_id":
  "d198379033af4d02aaa63c069e661fef", "roles": ["reader", "member"],
  "service_project_domain_id": "default", "service_project_id":
  "ad0ce459e9f9448e9632cd0df5cd5ccd", "service_roles": ["reader",
  "service", "admin", "member"], "service_user_domain_id": "default",
  "service_user_id": "0814687eb89f4de594732d1dc83e69ed", "system_scope":
  null, "user_domain_id": "default", "user_id":
  "2049143f1a2e4d39b38bed227226db61"} target={"backup_type": "daily",
  "base_image_ref": "22617963-7650-423c-8a9f-7572da6026e2",
  "boot_roles": "reader,member", "checksum": null, "container_format":
  "bare", "created_at": "2023-02-01T11:48:51.000000", "disk_format":
  "raw", "extra_properties": {}, "hw_cdrom_bus": "ide", "hw_disk_bus":
  "virtio", "hw_machine_type": "pc", "hw_video_model": "virtio",
  "hw_vif_model": "virtio", "image_id":
  "db4a3513-7ee7-43db-a184-44b482eb0fc2", "image_location": "snapshot",
  "image_state": "available", "image_type": "backup", "instance_uuid":
  "de308b29-2edd-4aee-886d-230ea41bd528", "member": null, "min_disk": 1,
  "min_ram": 0, "name": "tempest-backup-1-916792853", "os_hash_algo":
  null, "os_hash_value": null, "os_hidden": false, "owner":
  "d198379033af4d02aaa63c069e661fef", "owner_id":
  "d198379033af4d02aaa63c069e661fef", "owner_project_name": "tempest-
  ServerActionsTestJSON-1947886094", "owner_specified.openstack.md5":
  "", "owner_specified.openstack.object": "images/cirros-raw",
  "owner_specified.openstack.sha256": "", "owner_user_name": "tempest-
  ServerActionsTestJSON-1947886094-project", "project_id":
  "d198379033af4d02aaa63c069e661fef", "protected": false, "size":
  1073741824, "status": "active", "tags": [], "updated_at":
  "2023-02-01T11:48:57.000000", "user_id":
  "2049143f1a2e4d39b38bed227226db61", "virtual_size": null,
  "visibility": "private"} {{(pid=109935) enforce
  /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1036}}

  Feb 01 11:49:10.099258 np0032932858 glance-api[109934]: INFO
  eventlet.wsgi.server [None req-3bd97e63-562a-403e-8ca0-7e0da0f38c05
  tempest-ImageLocationsTest-1995906154 tempest-
  ImageLocationsTest-1995906154-project]
  149.202.176.133,149.202.176.133,149.202.176.133 - - [01/Feb/2023
  11:49:10] "PATCH /v2/images/ee8671ea-4771-461b-b14b-f4d5b4598496
  HTTP/1.1" 400 254 0.211937

  Example failed run:
  https://zuul.opendev.org/t/openstack/build/e57489f5e9484160b005c329b205f932/logs

  
  logsearch query (https://github.com/gibizer/zuul-log-search): 
  $ logsearch log --job nova-ceph-multistore --result FAILURE --file  controller/logs/screen-g-api.txt 'tempest-ImageLocationsTest.*The HTTP URL is invalid: .*RemoteDisconnected.*Remote end closed connection without response' --days 10
  [snip]:
  Builds with matching logs 5/25:
  +----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+
  | uuid                             | finished            | project        | pipeline | review                            | branch |
  +----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+
  | a9b9bf62802b4c9b9421b138c5c56024 | 2023-02-07T11:01:21 | openstack/nova | check    | https://review.opendev.org/872432 | master |
  | bc7699718de14023b3142c78a6e9f10c | 2023-02-02T22:24:16 | openstack/nova | check    | https://review.opendev.org/868236 | master |
  | a75c88f94d614eeea6bfa2fadada1638 | 2023-02-02T13:03:24 | openstack/nova | check    | https://review.opendev.org/854823 | master |
  | dc19fe23fd9849708b884915fcdfb086 | 2023-02-01T21:04:59 | openstack/nova | check    | https://review.opendev.org/872441 | master |
  | e57489f5e9484160b005c329b205f932 | 2023-02-01T12:15:19 | openstack/nova | gate     | https://review.opendev.org/872204 | master |
  +----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/2006473/+subscriptions