yahoo-eng-team team mailing list archive

[Abhishek Kekane <1914826@xxxxxxxxxxxxxxxxxx>]

** Changed in: glance
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1914826

Title:
  web-download with invalid url does not report error

Status in Glance:
  Fix Released

Bug description:
  In my testing, if I provide a URL to web-download that yields an error
  from urlopen(), I never see the store listed in the
  os_glance_failed_import list, and the store remains in
  os_glance_importing_to_stores. The image status does not change, which
  means there's no way for the API client to know that the import
  failed.

  I found this when debugging a gate issue where occasionally the
  tempest web-download test fails. It ends up waiting many minutes for
  the import to complete, even though it failed long before that. In
  that case, the cirros link we use for testing web-download raised a
  timeout.

  From my log, here what we log as returning to the user just before we
  start the import:

  Feb 05 20:18:02 guaranine devstack@g-api.service[1008592]: DEBUG
  oslo_policy.policy [-] enforce: rule="modify_image"
  creds={"domain_id": null, "is_admin_project": true,
  "project_domain_id": "default", "project_id":
  "59a5997403484e97803cac28b7aa7366", "roles": ["reader", "member"],
  "service_project_domain_id": null, "service_project_id": null,
  "service_roles": [], "service_user_domain_id": null,
  "service_user_id": null, "system_scope": null, "user_domain_id":
  "default", "user_id": "10e5d60c60e54ab3889bcd57e367fe01"}
  target={"checksum": null, "container_format": "bare", "created_at":
  "2021-02-05T20:18:03.000000", "disk_format": "raw",
  "extra_properties": {}, "image_id": "70917fce-
  bfc6-4d57-aa54-58235d09cf24", "locations": [], "min_disk": 0,
  "min_ram": 0, "name": "test", "os_glance_failed_import": "",
  "os_glance_import_task": "e2cb5441-8c92-45c6-9363-f4b7915401e1",
  "os_glance_importing_to_stores": "cheap", "os_hash_algo": null,
  "os_hash_value": null, "os_hidden": false, "owner":
  "59a5997403484e97803cac28b7aa7366", "protected": false, "size": null,
  "status": "importing", "tags": [], "updated_at":
  "2021-02-05T20:18:03.000000", "virtual_size": null, "visibility":
  "shared"} {{(pid=1008592) enforce /usr/local/lib/python3.8/dist-
  packages/oslo_policy/policy.py:994}}

  Note that os_glance_importing_to_stores="cheap" and
  os_glance_failed_import="". Shortly after this, the web-download task
  fails:

  Feb 05 20:18:03 guaranine devstack@g-api.service[1008592]: ERROR
  glance.async_.flows._internal_plugins.web_download [-] Task
  e2cb5441-8c92-45c6-9363-f4b7915401e1 failed with exception <urlopen
  error [Errno -2] Name or service not known>: urllib.error.URLError:
  <urlopen error [Errno -2] Name or service not known>

  Here's where the task is fully reverted:

  Feb 05 20:18:03 guaranine devstack@g-api.service[1008592]: WARNING glance.async_.taskflow_executor [-] Task 'api_image_import-WebDownlo
  ad-e2cb5441-8c92-45c6-9363-f4b7915401e1' (bc722b5c-ddd4-404b-9c09-8625ed9c5941) transitioned into state 'REVERTED' from state 'REVERTIN
  G' with result 'None'

  And after that, here's what we're still returning to the user:

  Feb 05 20:18:03 guaranine devstack@g-api.service[1008592]: DEBUG oslo_policy.policy [-] enforce: rule="get_image" creds={"domain_id": n
  ull, "is_admin_project": true, "project_domain_id": "default", "project_id": "59a5997403484e97803cac28b7aa7366", "roles": ["reader", "m
  ember"], "service_project_domain_id": null, "service_project_id": null, "service_roles": [], "service_user_domain_id": null, "service_u
  ser_id": null, "system_scope": null, "user_domain_id": "default", "user_id": "10e5d60c60e54ab3889bcd57e367fe01"} target={"checksum": nu
  ll, "container_format": "bare", "created_at": "2021-02-05T20:18:03.000000", "disk_format": "raw", "extra_properties": {}, "image_id": "
  70917fce-bfc6-4d57-aa54-58235d09cf24", "locations": [], "min_disk": 0, "min_ram": 0, "name": "test", "os_glance_failed_import": "", "os
  _glance_import_task": "e2cb5441-8c92-45c6-9363-f4b7915401e1", "os_glance_importing_to_stores": "cheap", "os_hash_algo": null, "os_hash_
  value": null, "os_hidden": false, "owner": "59a5997403484e97803cac28b7aa7366", "protected": false, "size": null, "status": "queued", "t
  ags": [], "updated_at": "2021-02-05T20:18:03.000000", "virtual_size": null, "visibility": "shared"} {{(pid=1008592) enforce /usr/local/
  lib/python3.8/dist-packages/oslo_policy/policy.py:994}}

  Note that os_glance_importing_to_stores="cheap" and
  os_glance_failed_import="". In this case, "cheap" should have moved
  from "importing" to "failed".

  I wrote a tempest negative test for this situation using a totally
  bogus URL, which is here:

  https://review.opendev.org/c/openstack/tempest/+/774303

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1914826/+subscriptions