← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 2025497] [NEW] Unable to delete image from glance cinder store

 

Public bug reported:

OpenStack master branch built using Devstack 6/30/23

Configuring glance to use cinder backends as multistores.

Configuration works as documented and images can be created in the
cinder backend, however when trying to delete any image from glance that
is located in a cinder store this will fail with the following:

stack@multi1:~$ glance image-delete 7934ed5e-9958-47e2-9a6a-5aab9a33dfca
'HTTP 500 Internal Server Error: The server has either erred or is incapable of performing the requested operation.': Unable to delete image '7934ed5e-9958-47e2-9a6a-5aab9a33dfca'

The image in question is:

stack@multi1:~$ glance image-show 7934ed5e-9958-47e2-9a6a-5aab9a33dfca
+----------------------------------+----------------------------------------------------------------------------------+
| Property                         | Value                                                                            |
+----------------------------------+----------------------------------------------------------------------------------+
| checksum                         | a3bf59112e4930ef32397be81c761a80                                                 |
| container_format                 | bare                                                                             |
| created_at                       | 2023-06-30T16:32:14Z                                                             |
| disk_format                      | raw                                                                              |
| id                               | 7934ed5e-9958-47e2-9a6a-5aab9a33dfca                                             |
| locations                        | [{"url": "cinder://pure2/7f299476-9245-4648-9e2e-a9cca62f9354", "metadata":      |
|                                  | {"store": "pure2"}}]                                                             |
| min_disk                         | 0                                                                                |
| min_ram                          | 0                                                                                |
| name                             | Ubuntu 22.04                                                                     |
| os_hash_algo                     | sha512                                                                           |
| os_hash_value                    | 0bbfdb7382c8767f60d8bb6da703819dc6fdaef36e6e544740eb5fffcb677d11ba23c3eed9e885b8 |
|                                  | b82d8444a7adcfd8712c2dc4dcb8f9589f5a9787a6879aa1                                 |
| os_hidden                        | False                                                                            |
| owner                            | 6f9ddcf4d9f148789b03a20447a39f8c                                                 |
| owner_specified.openstack.md5    |                                                                                  |
| owner_specified.openstack.object | images/Ubuntu 22.04                                                              |
| owner_specified.openstack.sha256 |                                                                                  |
| protected                        | False                                                                            |
| size                             | 2361393152                                                                       |
| status                           | active                                                                           |
| stores                           | pure2                                                                            |
| tags                             | []                                                                               |
| updated_at                       | 2023-06-30T16:33:00Z                                                             |
| virtual_size                     | 2361393152                                                                       |
| visibility                       | public                                                                           |
+----------------------------------+----------------------------------------------------------------------------------+

The error in the glance-api log file is:

Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi [None req-2e0ef496-2877-431c-af02-fe439fe33af2 admin admin] Caught error: Deleting images from this store is not supported.: glance_store.exceptions.StoreDeleteNotSupported: Deleting images from this store is not supported.
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi Traceback (most recent call last):
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/wsgi.py", line 1295, in __call__
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     action_result = self.dispatch(self.controller, action,
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/wsgi.py", line 1338, in dispatch
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     return method(*args, **kwargs)
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/utils.py", line 414, in wrapped
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     return func(self, req, *args, **kwargs)
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/api/v2/images.py", line 882, in delete
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     staging_store.delete(loc)
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/glance_store/capabilities.py", line 174, in op_checker
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     raise op_exec_map[op](**kwargs)
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi glance_store.exceptions.StoreDeleteNotSupported: Deleting images from this store is not supported.
Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi


The contents of glance-api.conf are:

[DEFAULT]
use_keystone_limits = True
enforce_secure_rbac = true
worker_self_reference_url = http://127.0.0.1:60999
logging_exception_prefix = ERROR %(name)s %(instance)s
logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s
logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s
logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}}
public_endpoint = http://x.x.x.x/image
show_multiple_locations = True
show_image_direct_url = False
transport_url = rabbit://stackrabbit:pureuser@x.x.x.x:5672/
image_cache_dir = /opt/stack/data/glance/cache/
use_syslog = False
debug = True
enabled_backends = pure1:cinder, pure2:cinder, pure3:cinder, pure4:cinder

[database]
connection = mysql+pymysql://root:pureuser@127.0.0.1/glance?charset=utf8&plugin=dbcounter

[oslo_concurrency]
lock_path = /opt/stack/data/glance/locks

[paste_deploy]
flavor = keystone+cachemanagement

[keystone_authtoken]
memcached_servers = localhost:11211
cafile = /opt/stack/data/ca-bundle.pem
project_domain_name = Default
project_name = service
user_domain_name = Default
password = *******
username = glance
auth_url = http://x.x.x.x/identity
interface = public
auth_type = password

[oslo_messaging_notifications]
driver = messagingv2

[glance_store]
filesystem_store_datadir = /opt/stack/data/glance/images/
stores = file,http,cinder
default_store = cinder
default_backend = pure2

[cors]
allowed_origin = http://x.x.x.x

[oslo_policy]
enforce_new_defaults = true
enforce_scope = true

[oslo_limit]
endpoint_id = a63e27f2e4cb4026a0bd5ab74eaa46be
system_scope = all
auth_url = http://x.x.x.x/identity
auth_type = password
username = glance
password = *******
user_domain_name = Default

[pure1]
store_description = Pure
cinder_store_auth_address = http://x.x.x.x/identity
cinder_store_user_name = glance
cinder_store_password = *******
cinder_store_project_name = service
cinder_volume_type = pure

[pure2]
store_description = Pure
cinder_store_auth_address = http://x.x.x.x/identity
cinder_store_user_name = glance
cinder_store_password = *******
cinder_store_project_name = service
cinder_volume_type = pure2


I understood that HTTP stores were read-only, but you should definitely be able to delete an image from a cinder backend store - it is only a volume in the end.

** Affects: glance
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/2025497

Title:
  Unable to delete image from glance cinder store

Status in Glance:
  New

Bug description:
  OpenStack master branch built using Devstack 6/30/23

  Configuring glance to use cinder backends as multistores.

  Configuration works as documented and images can be created in the
  cinder backend, however when trying to delete any image from glance
  that is located in a cinder store this will fail with the following:

  stack@multi1:~$ glance image-delete 7934ed5e-9958-47e2-9a6a-5aab9a33dfca
  'HTTP 500 Internal Server Error: The server has either erred or is incapable of performing the requested operation.': Unable to delete image '7934ed5e-9958-47e2-9a6a-5aab9a33dfca'

  The image in question is:

  stack@multi1:~$ glance image-show 7934ed5e-9958-47e2-9a6a-5aab9a33dfca
  +----------------------------------+----------------------------------------------------------------------------------+
  | Property                         | Value                                                                            |
  +----------------------------------+----------------------------------------------------------------------------------+
  | checksum                         | a3bf59112e4930ef32397be81c761a80                                                 |
  | container_format                 | bare                                                                             |
  | created_at                       | 2023-06-30T16:32:14Z                                                             |
  | disk_format                      | raw                                                                              |
  | id                               | 7934ed5e-9958-47e2-9a6a-5aab9a33dfca                                             |
  | locations                        | [{"url": "cinder://pure2/7f299476-9245-4648-9e2e-a9cca62f9354", "metadata":      |
  |                                  | {"store": "pure2"}}]                                                             |
  | min_disk                         | 0                                                                                |
  | min_ram                          | 0                                                                                |
  | name                             | Ubuntu 22.04                                                                     |
  | os_hash_algo                     | sha512                                                                           |
  | os_hash_value                    | 0bbfdb7382c8767f60d8bb6da703819dc6fdaef36e6e544740eb5fffcb677d11ba23c3eed9e885b8 |
  |                                  | b82d8444a7adcfd8712c2dc4dcb8f9589f5a9787a6879aa1                                 |
  | os_hidden                        | False                                                                            |
  | owner                            | 6f9ddcf4d9f148789b03a20447a39f8c                                                 |
  | owner_specified.openstack.md5    |                                                                                  |
  | owner_specified.openstack.object | images/Ubuntu 22.04                                                              |
  | owner_specified.openstack.sha256 |                                                                                  |
  | protected                        | False                                                                            |
  | size                             | 2361393152                                                                       |
  | status                           | active                                                                           |
  | stores                           | pure2                                                                            |
  | tags                             | []                                                                               |
  | updated_at                       | 2023-06-30T16:33:00Z                                                             |
  | virtual_size                     | 2361393152                                                                       |
  | visibility                       | public                                                                           |
  +----------------------------------+----------------------------------------------------------------------------------+

  The error in the glance-api log file is:

  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi [None req-2e0ef496-2877-431c-af02-fe439fe33af2 admin admin] Caught error: Deleting images from this store is not supported.: glance_store.exceptions.StoreDeleteNotSupported: Deleting images from this store is not supported.
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi Traceback (most recent call last):
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/wsgi.py", line 1295, in __call__
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     action_result = self.dispatch(self.controller, action,
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/wsgi.py", line 1338, in dispatch
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     return method(*args, **kwargs)
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/common/utils.py", line 414, in wrapped
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     return func(self, req, *args, **kwargs)
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/opt/stack/glance/glance/api/v2/images.py", line 882, in delete
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     staging_store.delete(loc)
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi   File "/usr/local/lib/python3.10/dist-packages/glance_store/capabilities.py", line 174, in op_checker
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi     raise op_exec_map[op](**kwargs)
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi glance_store.exceptions.StoreDeleteNotSupported: Deleting images from this store is not supported.
  Jun 30 17:54:45 multi1 devstack@g-api.service[1052009]: ERROR glance.common.wsgi

  
  The contents of glance-api.conf are:

  [DEFAULT]
  use_keystone_limits = True
  enforce_secure_rbac = true
  worker_self_reference_url = http://127.0.0.1:60999
  logging_exception_prefix = ERROR %(name)s %(instance)s
  logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s
  logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s
  logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}}
  public_endpoint = http://x.x.x.x/image
  show_multiple_locations = True
  show_image_direct_url = False
  transport_url = rabbit://stackrabbit:pureuser@x.x.x.x:5672/
  image_cache_dir = /opt/stack/data/glance/cache/
  use_syslog = False
  debug = True
  enabled_backends = pure1:cinder, pure2:cinder, pure3:cinder, pure4:cinder

  [database]
  connection = mysql+pymysql://root:pureuser@127.0.0.1/glance?charset=utf8&plugin=dbcounter

  [oslo_concurrency]
  lock_path = /opt/stack/data/glance/locks

  [paste_deploy]
  flavor = keystone+cachemanagement

  [keystone_authtoken]
  memcached_servers = localhost:11211
  cafile = /opt/stack/data/ca-bundle.pem
  project_domain_name = Default
  project_name = service
  user_domain_name = Default
  password = *******
  username = glance
  auth_url = http://x.x.x.x/identity
  interface = public
  auth_type = password

  [oslo_messaging_notifications]
  driver = messagingv2

  [glance_store]
  filesystem_store_datadir = /opt/stack/data/glance/images/
  stores = file,http,cinder
  default_store = cinder
  default_backend = pure2

  [cors]
  allowed_origin = http://x.x.x.x

  [oslo_policy]
  enforce_new_defaults = true
  enforce_scope = true

  [oslo_limit]
  endpoint_id = a63e27f2e4cb4026a0bd5ab74eaa46be
  system_scope = all
  auth_url = http://x.x.x.x/identity
  auth_type = password
  username = glance
  password = *******
  user_domain_name = Default

  [pure1]
  store_description = Pure
  cinder_store_auth_address = http://x.x.x.x/identity
  cinder_store_user_name = glance
  cinder_store_password = *******
  cinder_store_project_name = service
  cinder_volume_type = pure

  [pure2]
  store_description = Pure
  cinder_store_auth_address = http://x.x.x.x/identity
  cinder_store_user_name = glance
  cinder_store_password = *******
  cinder_store_project_name = service
  cinder_volume_type = pure2

  
  I understood that HTTP stores were read-only, but you should definitely be able to delete an image from a cinder backend store - it is only a volume in the end.

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/2025497/+subscriptions



Follow ups