yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2070258] Re: Delete image call fails with 500 during image download is in progress

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Pranali Deore <2070258@xxxxxxxxxxxxxxxxxx>
Date: Thu, 27 Jun 2024 13:13:32 -0000
Reply-to: Bug 2070258 <2070258@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
** Project changed: glance => glance-store

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/2070258

Title:
  Delete image call fails with 500 during image download is in progress

Status in glance_store:
  In Progress

Bug description:
  When glance is using cinder backend, and during image download if
  delete call for the same image has been attempted, delete call fails
  'HTTP 500 Internal Server Error: The server has either erred or is
  incapable of performing the requested operation.': Unable to delete
  image 'cf51102f-89a1-481d-adfc-2aa02b641d02' which should give 409
  HTTPConflict error to the user.

  Steps to reproduce:

  1. Create Image
  $ glance image-create-via-import --container-format bare --disk-format qcow2 --name test-img --import-method web-download --uri https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-aarch64-disk.img --store lvmdriver-1

  2. Download the Image once it is active and simultaneously delete the image in other terminal
  $ glance image-download 6a63cfe6-c10c-4319-8072-44db1012c6c3 --file test.txt

  $ glance image-delete 6a63cfe6-c10c-4319-8072-44db1012c6c3
  'HTTP 500 Internal Server Error: The server has either erred or is incapable of performing the requested operation.': Unable to delete image '6a63cfe6-c10c-4319-8072-44db1012c6c3'


  g-api Log:

  
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi [None req-78ab744d-169d-44c2-ae6a-fcd74caf2e4a admin admin] Caught error: Failed to delete volume b54a9b0f-758d-481a-904e-a450edc73c50: Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots, awaiting a transfer, or be disassociated from snapshots after volume transfer. (HTTP 400) (Request-ID: req-68dd17d7-5013-40f9-8052-06f195b8033f): glance_store.exceptions.BackendException: Failed to delete volume b54a9b0f-758d-481a-904e-a450edc73c50: Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots, awaiting a transfer, or be disassociated from snapshots after volume transfer. (HTTP 400) (Request-ID: req-68dd17d7-5013-40f9-8052-06f195b8033f)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi Traceback (most recent call last):
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/glance_store/_drivers/cinder/store.py", line 1123, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     self.volume_api.delete(client, loc.volume_id)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/glance_store/common/cinder_utils.py", line 71, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     client.volumes.delete(volume_id)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     return self._delete(loc)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 313, in _delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     resp, body = self.api.client.delete(url)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 229, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     return self._cs_request(url, 'DELETE', **kwargs)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     return self.request(url, method, **kwargs)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     raise exceptions.from_response(resp, body)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots, awaiting a transfer, or be disassociated from snapshots after volume transfer. (HTTP 400) (Request-ID: req-68dd17d7-5013-40f9-8052-06f195b8033f)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi 
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi During handling of the above exception, another exception occurred:
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi 
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi Traceback (most recent call last):
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/wsgi.py", line 1302, in __call__
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     action_result = self.dispatch(self.controller, action,
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/wsgi.py", line 1345, in dispatch
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     return method(*args, **kwargs)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/utils.py", line 415, in wrapped
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     return func(self, req, *args, **kwargs)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/api/v2/images.py", line 906, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     image.delete()
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/domain/proxy.py", line 200, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     self.base.delete()
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/domain/proxy.py", line 200, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     self.base.delete()
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/location.py", line 458, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     self.store_utils.delete_image_location_from_backend(
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/store_utils.py", line 130, in delete_image_location_from_backend
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     safe_delete_from_backend(context, image_id, location)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/store_utils.py", line 57, in safe_delete_from_backend
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     ret = store_api.delete(location['url'],
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/glance_store/multi_backend.py", line 510, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     return store.delete(loc, context=context)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/glance_store/capabilities.py", line 176, in op_checker
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     return store_op_fun(store, *args, **kwargs)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/glance_store/_drivers/cinder/store.py", line 1129, in delete
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi     raise exceptions.BackendException(msg)
  Jun 24 10:35:02 pranali-new-devstack devstack@g-api.service[2485343]: ERROR glance.common.wsgi glance_store.exceptions.BackendException: Failed to delete volume b54a9b0f-758d-481a-904e-a450edc73c50: Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots, awaiting a transfer, or be disassociated from snapshots after volume transfer. (HTTP 400) (Request-ID: req-68dd17d7-5013-40f9-8052-06f195b8033f)

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance-store/+bug/2070258/+subscriptions
References

[Bug 2070258] [NEW] Delete image call fails with 500 during image download is in progress
From: Pranali Deore, 2024-06-24