yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2120834] [NEW] some rally jobs broken with glance AttributeError: module 'sqlalchemy.sql' has no attribute 'event'. Did you mean: 'events'?

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: yatin <2120834@xxxxxxxxxxxxxxxxxx>
Date: Mon, 18 Aug 2025 11:01:28 -0000
Reply-to: Bug 2120834 <2120834@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Traceback:-

Aug 11 10:23:32.546603 np0c3b18b4f54e4 devstack@g-api.service[71147]: [pid: 71147|app: -1|req: -1/1] 127.0.0.1 () {34 vars in 482 bytes} [Mon Aug 11 10:23:32 2025] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0)
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: CRITICAL glance [None req-a5c99058-2b17-4f6a-a588-2e2e81e3c244 None None] Unhandled error: AttributeError: module 'sqlalchemy.sql' has no attribute 'event'. Did you mean: 'events'?
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance Traceback (most recent call last):
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/wsgi/api.py", line 22, in <module>
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     application = wsgi_app.init_app()
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance                   ^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/common/wsgi_app.py", line 160, in init_app
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return config.load_paste_app('glance-api')
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/common/config.py", line 753, in load_paste_app
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     app = deploy.loadapp("config:%s" % conf_file, name=app_name)
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 246, in loadapp
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return loadobj(APP, uri, name=name, **kw)
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return context.create()
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.object_type.invoke(self)
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 129, in invoke
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return fix_call(
Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     val = callable(*args, **kw)
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/__init__.py", line 23, in root_app_factory
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return paste.urlmap.urlmap_factory(loader, global_conf, **local_conf)
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/urlmap.py", line 26, in urlmap_factory
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     app = loader.get_app(app_name, global_conf=global_conf)
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 352, in get_app
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.app_context(name=name, global_conf=global_conf).create()
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.object_type.invoke(self)
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 129, in invoke
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return fix_call(
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     val = callable(*args, **kw)
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/__init__.py", line 31, in pipeline_factory
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     app = loader.get_app(pipeline[-1])
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 352, in get_app
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.app_context(name=name, global_conf=global_conf).create()
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.object_type.invoke(self)
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 129, in invoke
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return fix_call(
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     val = callable(*args, **kw)
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/__init__.py", line 23, in root_app_factory
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return paste.urlmap.urlmap_factory(loader, global_conf, **local_conf)
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/urlmap.py", line 26, in urlmap_factory
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     app = loader.get_app(app_name, global_conf=global_conf)
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 352, in get_app
Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.app_context(name=name, global_conf=global_conf).create()
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.object_type.invoke(self)
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 136, in invoke
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return fix_call(context.object, context.global_conf, **context.local_conf)
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     val = callable(*args, **kw)
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/common/wsgi.py", line 866, in factory
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return cls(APIMapper())
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/v2/router.py", line 622, in __init__
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     cache_manage_resource = cached_images.create_resource()
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/v2/cached_images.py", line 307, in create_resource
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return wsgi.Resource(CacheController(), deserializer, serializer)
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance                          ^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/v2/cached_images.py", line 55, in __init__
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.cache = image_cache.ImageCache()
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance                  ^^^^^^^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/image_cache/__init__.py", line 170, in __init__
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.init_driver()
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/image_cache/__init__.py", line 191, in init_driver
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.configure_driver()
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/image_cache/__init__.py", line 200, in configure_driver
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.driver.configure()
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/image_cache/drivers/centralized_db.py", line 68, in configure
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.db_api.node_reference_create(
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/common/utils.py", line 542, in wrapper
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return f(*args, **kwargs)
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 2340, in node_reference_create
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     with session_for_write() as session:
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return next(self.gen)
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/osprofiler/osprofiler/sqlalchemy.py", line 58, in wrap_session
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     add_tracing(sqlalchemy, s.bind, "db")
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/osprofiler/osprofiler/sqlalchemy.py", line 45, in add_tracing
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     sqlalchemy.event.listen(engine, "before_cursor_execute",
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     ^^^^^^^^^^^^^^^^
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance AttributeError: module 'sqlalchemy.sql' has no attribute 'event'. Did you mean: 'events'?
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance 
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: unable to load app 0 (mountpoint='') (callable not found or import error)
Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: *** no app loaded. going in full dynamic mode ***

Example failure https://3b030092da707063c3cb-
fffe026581e4ceffd7fb8bb5725948b2.ssl.cf1.rackcdn.com/openstack/6fb9ede03bd245c59fef3d07536c2d0c/controller/logs/screen-
g-api.txt

Builds:- https://zuul.opendev.org/t/openstack/builds?job_name=neutron-
ovs-rally-task&job_name=cinder-rally-task&skip=0

Looks triggered by
https://review.opendev.org/c/openstack/glance/+/926560

** Affects: glance
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/2120834

Title:
  some rally jobs broken with glance AttributeError: module
  'sqlalchemy.sql' has no attribute 'event'. Did you mean: 'events'?

Status in Glance:
  New

Bug description:
  Traceback:-

  Aug 11 10:23:32.546603 np0c3b18b4f54e4 devstack@g-api.service[71147]: [pid: 71147|app: -1|req: -1/1] 127.0.0.1 () {34 vars in 482 bytes} [Mon Aug 11 10:23:32 2025] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0)
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: CRITICAL glance [None req-a5c99058-2b17-4f6a-a588-2e2e81e3c244 None None] Unhandled error: AttributeError: module 'sqlalchemy.sql' has no attribute 'event'. Did you mean: 'events'?
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance Traceback (most recent call last):
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/wsgi/api.py", line 22, in <module>
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     application = wsgi_app.init_app()
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance                   ^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/common/wsgi_app.py", line 160, in init_app
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return config.load_paste_app('glance-api')
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/common/config.py", line 753, in load_paste_app
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     app = deploy.loadapp("config:%s" % conf_file, name=app_name)
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 246, in loadapp
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return loadobj(APP, uri, name=name, **kw)
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return context.create()
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.object_type.invoke(self)
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 129, in invoke
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return fix_call(
  Aug 11 10:23:32.546683 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     val = callable(*args, **kw)
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/__init__.py", line 23, in root_app_factory
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return paste.urlmap.urlmap_factory(loader, global_conf, **local_conf)
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/urlmap.py", line 26, in urlmap_factory
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     app = loader.get_app(app_name, global_conf=global_conf)
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 352, in get_app
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.app_context(name=name, global_conf=global_conf).create()
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.object_type.invoke(self)
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 129, in invoke
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return fix_call(
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     val = callable(*args, **kw)
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.547996 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/__init__.py", line 31, in pipeline_factory
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     app = loader.get_app(pipeline[-1])
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 352, in get_app
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.app_context(name=name, global_conf=global_conf).create()
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.object_type.invoke(self)
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 129, in invoke
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return fix_call(
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     val = callable(*args, **kw)
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/__init__.py", line 23, in root_app_factory
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return paste.urlmap.urlmap_factory(loader, global_conf, **local_conf)
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/urlmap.py", line 26, in urlmap_factory
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     app = loader.get_app(app_name, global_conf=global_conf)
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 352, in get_app
  Aug 11 10:23:32.548971 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.app_context(name=name, global_conf=global_conf).create()
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return self.object_type.invoke(self)
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 136, in invoke
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return fix_call(context.object, context.global_conf, **context.local_conf)
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/data/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     val = callable(*args, **kw)
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance           ^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/common/wsgi.py", line 866, in factory
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return cls(APIMapper())
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/v2/router.py", line 622, in __init__
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     cache_manage_resource = cached_images.create_resource()
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/v2/cached_images.py", line 307, in create_resource
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return wsgi.Resource(CacheController(), deserializer, serializer)
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance                          ^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/api/v2/cached_images.py", line 55, in __init__
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.cache = image_cache.ImageCache()
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance                  ^^^^^^^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.550072 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/image_cache/__init__.py", line 170, in __init__
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.init_driver()
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/image_cache/__init__.py", line 191, in init_driver
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.configure_driver()
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/image_cache/__init__.py", line 200, in configure_driver
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.driver.configure()
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/image_cache/drivers/centralized_db.py", line 68, in configure
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     self.db_api.node_reference_create(
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/common/utils.py", line 542, in wrapper
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return f(*args, **kwargs)
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 2340, in node_reference_create
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     with session_for_write() as session:
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     return next(self.gen)
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance            ^^^^^^^^^^^^^^
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/osprofiler/osprofiler/sqlalchemy.py", line 58, in wrap_session
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     add_tracing(sqlalchemy, s.bind, "db")
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance   File "/opt/stack/osprofiler/osprofiler/sqlalchemy.py", line 45, in add_tracing
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     sqlalchemy.event.listen(engine, "before_cursor_execute",
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance     ^^^^^^^^^^^^^^^^
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance AttributeError: module 'sqlalchemy.sql' has no attribute 'event'. Did you mean: 'events'?
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: ERROR glance 
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: unable to load app 0 (mountpoint='') (callable not found or import error)
  Aug 11 10:23:32.551252 np0c3b18b4f54e4 devstack@g-api.service[71148]: *** no app loaded. going in full dynamic mode ***

  Example failure https://3b030092da707063c3cb-
  fffe026581e4ceffd7fb8bb5725948b2.ssl.cf1.rackcdn.com/openstack/6fb9ede03bd245c59fef3d07536c2d0c/controller/logs/screen-
  g-api.txt

  Builds:- https://zuul.opendev.org/t/openstack/builds?job_name=neutron-
  ovs-rally-task&job_name=cinder-rally-task&skip=0

  Looks triggered by
  https://review.opendev.org/c/openstack/glance/+/926560

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/2120834/+subscriptions
Follow ups

[Bug 2120834] Re: some rally jobs broken with glance AttributeError: module 'sqlalchemy.sql' has no attribute 'event'. Did you mean: 'events'?
From: OpenStack Infra, 2025-08-19