yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #88270
[Bug 1953063] Re: Image import causes SQL type casting error on PostgreSQL
Reviewed: https://review.opendev.org/c/openstack/glance/+/820247
Committed: https://opendev.org/openstack/glance/commit/753c74c343ab4ef32879a3dd95c3a956af77f869
Submitter: "Zuul (22348)"
Branch: master
commit 753c74c343ab4ef32879a3dd95c3a956af77f869
Author: Dan Smith <dansmith@xxxxxxxxxx>
Date: Thu Dec 2 12:01:17 2021 -0800
Fix set_property_atomic() boolean type casting
In set_property_atomic() we pass an integer query filter for the
deleted field, since most projects use an integer for this column.
However, in glance the column is a boolean, which trips up postgres
since the types are different (mysql and sqlite work fine). This
minor change to use False instead of 0 should fix that for postgres
users.
Change-Id: I5149df76943c1c19f3204b904c0e2d3ef846bdf7
Closes-Bug: #1953063
** Changed in: glance
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1953063
Title:
Image import causes SQL type casting error on PostgreSQL
Status in Glance:
Fix Released
Bug description:
When doing an image-create and then an image-import via
# glance image-import --import-method web-download --uri
https://cloud.debian.org/images/cloud/OpenStack/current-10/debian-10-openstack-
amd64.raw ad4d6763-1156-4be3-a1d1-27decfea4eab
on a PostgreSQL database being used, a type casting exception occurs.
The same issue naturally occurs with
# glance image-create-via-import
as well:
--- cut ---
2021-12-02 18:03:04.407 12940 INFO eventlet.wsgi.server [req-fc8ff4ad-43ed-4835-b9e1-92e407a2999a d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] 10.200.9.103,127.0.0.1 - - [02/Dec/2021 18:03:04] "GET /v2/info/import HTTP/1.1" 200 338 1.595783
2021-12-02 18:03:04.414 12940 DEBUG glance.api.middleware.version_negotiation [req-fc8ff4ad-43ed-4835-b9e1-92e407a2999a d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Determining version of request: GET /v2/images/ad4d6763-1156-4be3-a1d1-27decfea4eab Accept: */* process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:45
2021-12-02 18:03:04.414 12940 DEBUG glance.api.middleware.version_negotiation [req-fc8ff4ad-43ed-4835-b9e1-92e407a2999a d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Using url versioning process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:57
2021-12-02 18:03:04.415 12940 DEBUG glance.api.middleware.version_negotiation [req-fc8ff4ad-43ed-4835-b9e1-92e407a2999a d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Matched version: v2 process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:69
2021-12-02 18:03:04.415 12940 DEBUG glance.api.middleware.version_negotiation [req-fc8ff4ad-43ed-4835-b9e1-92e407a2999a d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] new path /v2/images/ad4d6763-1156-4be3-a1d1-27decfea4eab process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:70
2021-12-02 18:03:04.445 12940 DEBUG glance.api.v2.images [req-1c832527-7bce-4fd9-a92d-223fbd13dfac d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] The 'locations' list of image ad4d6763-1156-4be3-a1d1-27decfea4eab is empty, not including 'direct_url' in response _format_image /usr/lib/python3/dist-packages/glance/api/v2/images.py:1337
2021-12-02 18:03:04.447 12940 INFO eventlet.wsgi.server [req-1c832527-7bce-4fd9-a92d-223fbd13dfac d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] 10.200.9.103,127.0.0.1 - - [02/Dec/2021 18:03:04] "GET /v2/images/ad4d6763-1156-4be3-a1d1-27decfea4eab HTTP/1.1" 200 826 0.033851
2021-12-02 18:03:04.452 12940 DEBUG glance.api.middleware.version_negotiation [req-1c832527-7bce-4fd9-a92d-223fbd13dfac d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Determining version of request: GET /v2/schemas/image Accept: */* process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:45
2021-12-02 18:03:04.453 12940 DEBUG glance.api.middleware.version_negotiation [req-1c832527-7bce-4fd9-a92d-223fbd13dfac d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Using url versioning process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:57
2021-12-02 18:03:04.453 12940 DEBUG glance.api.middleware.version_negotiation [req-1c832527-7bce-4fd9-a92d-223fbd13dfac d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Matched version: v2 process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:69
2021-12-02 18:03:04.454 12940 DEBUG glance.api.middleware.version_negotiation [req-1c832527-7bce-4fd9-a92d-223fbd13dfac d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] new path /v2/schemas/image process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:70
2021-12-02 18:03:04.457 12940 INFO eventlet.wsgi.server [req-ac24af9d-6857-47c5-9fdc-3e19e9daecf7 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] 10.200.9.103,127.0.0.1 - - [02/Dec/2021 18:03:04] "GET /v2/schemas/image HTTP/1.1" 200 6295 0.004862
2021-12-02 18:03:04.463 12940 DEBUG glance.api.middleware.version_negotiation [req-ac24af9d-6857-47c5-9fdc-3e19e9daecf7 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Determining version of request: POST /v2/images/ad4d6763-1156-4be3-a1d1-27decfea4eab/import Accept: */* process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:45
2021-12-02 18:03:04.463 12940 DEBUG glance.api.middleware.version_negotiation [req-ac24af9d-6857-47c5-9fdc-3e19e9daecf7 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Using url versioning process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:57
2021-12-02 18:03:04.464 12940 DEBUG glance.api.middleware.version_negotiation [req-ac24af9d-6857-47c5-9fdc-3e19e9daecf7 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Matched version: v2 process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:69
2021-12-02 18:03:04.464 12940 DEBUG glance.api.middleware.version_negotiation [req-ac24af9d-6857-47c5-9fdc-3e19e9daecf7 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] new path /v2/images/ad4d6763-1156-4be3-a1d1-27decfea4eab/import process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:70
2021-12-02 18:03:04.487 12940 DEBUG glance_store.multi_backend [req-09146ff0-4c9d-4036-ae6e-dabf9e366305 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Attempting to import store rbd _load_multi_store /usr/lib/python3/dist-packages/glance_store/multi_backend.py:168
2021-12-02 18:03:04.555 12940 DEBUG glance_store.capabilities [req-09146ff0-4c9d-4036-ae6e-dabf9e366305 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Store glance_store._drivers.rbd.Store doesn't support updating dynamic storage capabilities. Please overwrite 'update_capabilities' method of the store to implement updating logics if needed. update_capabilities /usr/lib/python3/dist-packages/glance_store/capabilities.py:95
2021-12-02 18:03:04.555 12940 DEBUG glance_store.driver [req-09146ff0-4c9d-4036-ae6e-dabf9e366305 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Late loading location class glance_store._drivers.rbd.StoreLocation get_store_location_class /usr/lib/python3/dist-packages/glance_store/driver.py:117
2021-12-02 18:03:04.556 12940 DEBUG glance_store.location [req-09146ff0-4c9d-4036-ae6e-dabf9e366305 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Registering scheme rbd with {'image_store_az1': {'store': <glance_store._drivers.rbd.Store object at 0x7f3091e466a0>, 'location_class': <class 'glance_store._drivers.rbd.StoreLocation'>, 'store_entry': 'rbd'}} register_scheme_backend_map /usr/lib/python3/dist-packages/glance_store/location.py:132
2021-12-02 18:03:04.570 12940 WARNING oslo_db.sqlalchemy.exc_filters [req-09146ff0-4c9d-4036-ae6e-dabf9e366305 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] DBAPIError exception wrapped.: psycopg2.errors.UndefinedFunction: operator does not exist: boolean <> integer
LINE 1: ...56-4be3-a1d1-27decfea4eab' AND image_properties.deleted != 0
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters cursor, statement, parameters, context
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters)
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters psycopg2.errors.UndefinedFunction: operator does not exist: boolean <> integer
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters LINE 1: ...56-4be3-a1d1-27decfea4eab' AND image_properties.deleted != 0
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters ^
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters
2021-12-02 18:03:04.570 12940 ERROR oslo_db.sqlalchemy.exc_filters
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi [req-09146ff0-4c9d-4036-ae6e-dabf9e366305 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] Caught error: (psycopg2.errors.UndefinedFunction) operator does not exist: boolean <> integer
LINE 1: ...56-4be3-a1d1-27decfea4eab' AND image_properties.deleted != 0
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: UPDATE image_properties SET updated_at=%(updated_at)s, deleted=%(deleted)s, value=%(value)s WHERE image_properties.name = %(name_1)s AND image_properties.image_id = %(image_id_1)s AND image_properties.deleted != %(deleted_1)s]
[parameters: {'updated_at': datetime.datetime(2021, 12, 2, 18, 3, 4, 565032), 'deleted': False, 'value': '3822c682-9fb5-443a-8788-63e65e0a769b', 'name_1': 'os_glance_import_task', 'image_id_1': 'ad4d6763-1156-4be3-a1d1-27decfea4eab', 'deleted_1': 0}]
(Background on this error at: http://sqlalche.me/e/f405): oslo_db.exception.DBError: (psycopg2.errors.UndefinedFunction) operator does not exist: boolean <> integer
LINE 1: ...56-4be3-a1d1-27decfea4eab' AND image_properties.deleted != 0
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: UPDATE image_properties SET updated_at=%(updated_at)s, deleted=%(deleted)s, value=%(value)s WHERE image_properties.name = %(name_1)s AND image_properties.image_id = %(image_id_1)s AND image_properties.deleted != %(deleted_1)s]
[parameters: {'updated_at': datetime.datetime(2021, 12, 2, 18, 3, 4, 565032), 'deleted': False, 'value': '3822c682-9fb5-443a-8788-63e65e0a769b', 'name_1': 'os_glance_import_task', 'image_id_1': 'ad4d6763-1156-4be3-a1d1-27decfea4eab', 'deleted_1': 0}]
(Background on this error at: http://sqlalche.me/e/f405)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi Traceback (most recent call last):
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi cursor, statement, parameters, context
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi cursor.execute(statement, parameters)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi psycopg2.errors.UndefinedFunction: operator does not exist: boolean <> integer
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi LINE 1: ...56-4be3-a1d1-27decfea4eab' AND image_properties.deleted != 0
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi ^
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi The above exception was the direct cause of the following exception:
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi Traceback (most recent call last):
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/common/wsgi.py", line 1355, in __call__
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi request, **action_args)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/common/wsgi.py", line 1398, in dispatch
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi return method(*args, **kwargs)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/common/utils.py", line 416, in wrapped
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/api/v2/images.py", line 319, in import_image
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi import_task.task_id)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/domain/proxy.py", line 110, in set_property_atomic
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi self.base.set_property_atomic(item, name, value)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/domain/proxy.py", line 110, in set_property_atomic
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi self.base.set_property_atomic(item, name, value)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/domain/proxy.py", line 110, in set_property_atomic
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi self.base.set_property_atomic(item, name, value)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi [Previous line repeated 2 more times]
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/db/__init__.py", line 228, in set_property_atomic
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi image.image_id, name, value)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/glance/db/sqlalchemy/api.py", line 813, in image_set_property_atomic
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi value=value, deleted=0))
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 982, in execute
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi return meth(self, multiparams, params)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi return connection._execute_clauseelement(self, multiparams, params)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi distilled_params,
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi e, statement, parameters, cursor, context
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1474, in _handle_dbapi_exception
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi util.raise_from_cause(newraise, exc_info)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi reraise(type(exception), exception, tb=exc_tb, cause=cause)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi raise value.with_traceback(tb)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi cursor, statement, parameters, context
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi cursor.execute(statement, parameters)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi oslo_db.exception.DBError: (psycopg2.errors.UndefinedFunction) operator does not exist: boolean <> integer
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi LINE 1: ...56-4be3-a1d1-27decfea4eab' AND image_properties.deleted != 0
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi ^
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi [SQL: UPDATE image_properties SET updated_at=%(updated_at)s, deleted=%(deleted)s, value=%(value)s WHERE image_properties.name = %(name_1)s AND image_properties.image_id = %(image_id_1)s AND image_properties.deleted != %(deleted_1)s]
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi [parameters: {'updated_at': datetime.datetime(2021, 12, 2, 18, 3, 4, 565032), 'deleted': False, 'value': '3822c682-9fb5-443a-8788-63e65e0a769b', 'name_1': 'os_glance_import_task', 'image_id_1': 'ad4d6763-1156-4be3-a1d1-27decfea4eab', 'deleted_1': 0}]
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi (Background on this error at: http://sqlalche.me/e/f405)
2021-12-02 18:03:04.578 12940 ERROR glance.common.wsgi
2021-12-02 18:03:04.618 12940 INFO eventlet.wsgi.server [req-09146ff0-4c9d-4036-ae6e-dabf9e366305 d1bf30b4a41a426691c0869cc9edee5b f45f6fcaa7944480bb82e6dc943d8962 - default default] 10.200.9.103,127.0.0.1 - - [02/Dec/2021 18:03:04] "POST /v2/images/ad4d6763-1156-4be3-a1d1-27decfea4eab/import HTTP/1.1" 500 454 0.155172
--- cut ---
To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1953063/+subscriptions
References
