yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #91844
[Bug 2015847] Re: [sqlalchemy-20][FT] Functional job db missconfiguration with SQLAlchemy2.0
Reviewed: https://review.opendev.org/c/openstack/neutron/+/880867
Committed: https://opendev.org/openstack/neutron/commit/82029c2c514aaca5b9d45af46d2f7d945ff3099b
Submitter: "Zuul (22348)"
Branch: master
commit 82029c2c514aaca5b9d45af46d2f7d945ff3099b
Author: Rodolfo Alonso Hernandez <ralonsoh@xxxxxxxxxx>
Date: Thu Apr 20 02:13:05 2023 +0200
Use a writer context for the online alembic migrations
``DBConnection`` class does not retrieve the URL string
correctly and cannot connect to the database. This patch
replaces it with default writer context that any database
operation transaction uses in Neutron.
Closes-Bug: #2015847
Change-Id: I60523690bc170ec4ba81312ada0f2d83542cae67
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2015847
Title:
[sqlalchemy-20][FT] Functional job db missconfiguration with
SQLAlchemy2.0
Status in neutron:
Fix Released
Bug description:
There is (probably) some issue with configuration of the functional
tests job neutron-functional-with-sqlalchemy-master which runs in
periodic queue. It is failing every day with same error:
2023-04-11 03:57:17.753314 | controller | neutron.tests.functional.db.test_migrations.TestWalkMigrationsPostgreSQL.test_walk_versions
2023-04-11 03:57:17.753325 | controller | -------------------------------------------------------------------------------------------
2023-04-11 03:57:17.753336 | controller |
2023-04-11 03:57:17.753346 | controller | Captured traceback:
2023-04-11 03:57:17.753357 | controller | ~~~~~~~~~~~~~~~~~~~
2023-04-11 03:57:17.753368 | controller | Traceback (most recent call last):
2023-04-11 03:57:17.753379 | controller |
2023-04-11 03:57:17.753390 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
2023-04-11 03:57:17.753401 | controller | self._dbapi_connection = engine.raw_connection()
2023-04-11 03:57:17.753412 | controller |
2023-04-11 03:57:17.753429 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3275, in raw_connection
2023-04-11 03:57:17.753441 | controller | return self.pool.connect()
2023-04-11 03:57:17.753452 | controller |
2023-04-11 03:57:17.753463 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 455, in connect
2023-04-11 03:57:17.753474 | controller | return _ConnectionFairy._checkout(self)
2023-04-11 03:57:17.753492 | controller |
2023-04-11 03:57:17.753504 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1271, in _checkout
2023-04-11 03:57:17.753514 | controller | fairy = _ConnectionRecord.checkout(pool)
2023-04-11 03:57:17.753525 | controller |
2023-04-11 03:57:17.753536 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 719, in checkout
2023-04-11 03:57:17.753548 | controller | rec = pool._do_get()
2023-04-11 03:57:17.753558 | controller |
2023-04-11 03:57:17.753570 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get
2023-04-11 03:57:17.753581 | controller | with util.safe_reraise():
2023-04-11 03:57:17.753591 | controller |
2023-04-11 03:57:17.753602 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
2023-04-11 03:57:17.753613 | controller | raise exc_value.with_traceback(exc_tb)
2023-04-11 03:57:17.753624 | controller |
2023-04-11 03:57:17.753635 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 166, in _do_get
2023-04-11 03:57:17.753645 | controller | return self._create_connection()
2023-04-11 03:57:17.753656 | controller |
2023-04-11 03:57:17.753667 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 396, in _create_connection
2023-04-11 03:57:17.753678 | controller | return _ConnectionRecord(self)
2023-04-11 03:57:17.753689 | controller |
2023-04-11 03:57:17.753700 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 681, in __init__
2023-04-11 03:57:17.753711 | controller | self.__connect()
2023-04-11 03:57:17.753722 | controller |
2023-04-11 03:57:17.753733 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 905, in __connect
2023-04-11 03:57:17.753743 | controller | with util.safe_reraise():
2023-04-11 03:57:17.753754 | controller |
2023-04-11 03:57:17.753769 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
2023-04-11 03:57:17.753781 | controller | raise exc_value.with_traceback(exc_tb)
2023-04-11 03:57:17.753792 | controller |
2023-04-11 03:57:17.753803 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 901, in __connect
2023-04-11 03:57:17.753814 | controller | self.dbapi_connection = connection = pool._invoke_creator(self)
2023-04-11 03:57:17.753825 | controller |
2023-04-11 03:57:17.753836 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 636, in connect
2023-04-11 03:57:17.753847 | controller | return dialect.connect(*cargs, **cparams)
2023-04-11 03:57:17.753858 | controller |
2023-04-11 03:57:17.753868 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 579, in connect
2023-04-11 03:57:17.753879 | controller | return self.loaded_dbapi.connect(*cargs, **cparams)
2023-04-11 03:57:17.753890 | controller |
2023-04-11 03:57:17.753901 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect
2023-04-11 03:57:17.753912 | controller | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
2023-04-11 03:57:17.753923 | controller |
2023-04-11 03:57:17.753934 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/eventlet/support/psycopg2_patcher.py", line 46, in eventlet_wait_callback
2023-04-11 03:57:17.753945 | controller | state = conn.poll()
2023-04-11 03:57:17.753962 | controller |
2023-04-11 03:57:17.753974 | controller | psycopg2.OperationalError: FATAL: password authentication failed for user "openstack_citest"
2023-04-11 03:57:17.753985 | controller | connection to server at "localhost" (::1), port 5432 failed: FATAL: password authentication failed for user "openstack_citest"
2023-04-11 03:57:17.753997 | controller |
2023-04-11 03:57:17.754007 | controller |
2023-04-11 03:57:17.754018 | controller |
2023-04-11 03:57:17.754029 | controller | The above exception was the direct cause of the following exception:
2023-04-11 03:57:17.754040 | controller |
2023-04-11 03:57:17.754051 | controller |
2023-04-11 03:57:17.754062 | controller | Traceback (most recent call last):
2023-04-11 03:57:17.754073 | controller |
2023-04-11 03:57:17.754084 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
2023-04-11 03:57:17.754095 | controller | return f(self, *args, **kwargs)
2023-04-11 03:57:17.754105 | controller |
2023-04-11 03:57:17.754116 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/db/test_migrations.py", line 576, in test_walk_versions
2023-04-11 03:57:17.754128 | controller | migration.do_alembic_command(config, 'upgrade', dest)
2023-04-11 03:57:17.754138 | controller |
2023-04-11 03:57:17.754149 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/migration/cli.py", line 90, in do_alembic_command
2023-04-11 03:57:17.754160 | controller | getattr(alembic_command, cmd)(config, *args, **kwargs)
2023-04-11 03:57:17.754171 | controller |
2023-04-11 03:57:17.754188 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/alembic/command.py", line 378, in upgrade
2023-04-11 03:57:17.754200 | controller | script.run_env()
2023-04-11 03:57:17.754211 | controller |
2023-04-11 03:57:17.754222 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/alembic/script/base.py", line 569, in run_env
2023-04-11 03:57:17.754233 | controller | util.load_python_file(self.dir, "env.py")
2023-04-11 03:57:17.754244 | controller |
2023-04-11 03:57:17.754255 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
2023-04-11 03:57:17.754266 | controller | module = load_module_py(module_id, path)
2023-04-11 03:57:17.754277 | controller |
2023-04-11 03:57:17.754288 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
2023-04-11 03:57:17.754299 | controller | spec.loader.exec_module(module) # type: ignore
2023-04-11 03:57:17.754310 | controller |
2023-04-11 03:57:17.754321 | controller | File "<frozen importlib._bootstrap_external>", line 883, in exec_module
2023-04-11 03:57:17.754332 | controller |
2023-04-11 03:57:17.754343 | controller | File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2023-04-11 03:57:17.754354 | controller |
2023-04-11 03:57:17.754365 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/migration/alembic_migrations/env.py", line 120, in <module>
2023-04-11 03:57:17.754376 | controller | run_migrations_online()
2023-04-11 03:57:17.754387 | controller |
2023-04-11 03:57:17.754398 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/migration/alembic_migrations/env.py", line 106, in run_migrations_online
2023-04-11 03:57:17.754409 | controller | with DBConnection(neutron_config.database.connection, connection) as conn:
2023-04-11 03:57:17.754420 | controller |
2023-04-11 03:57:17.754431 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/migration/connection.py", line 32, in __enter__
2023-04-11 03:57:17.754442 | controller | self.engine = session.create_engine(self.connection_url)
2023-04-11 03:57:17.754453 | controller |
2023-04-11 03:57:17.754464 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/debtcollector/removals.py", line 260, in wrapper
2023-04-11 03:57:17.754475 | controller | return f(*args, **kwargs)
2023-04-11 03:57:17.754485 | controller |
2023-04-11 03:57:17.754496 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/debtcollector/renames.py", line 41, in decorator
2023-04-11 03:57:17.754517 | controller | return wrapped(*args, **kwargs)
2023-04-11 03:57:17.754529 | controller |
2023-04-11 03:57:17.754540 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py", line 257, in create_engine
2023-04-11 03:57:17.754551 | controller | test_conn = _test_connection(engine, max_retries, retry_interval)
2023-04-11 03:57:17.754562 | controller |
2023-04-11 03:57:17.754573 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py", line 453, in _test_connection
2023-04-11 03:57:17.754584 | controller | return engine.connect()
2023-04-11 03:57:17.754595 | controller |
2023-04-11 03:57:17.754606 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3251, in connect
2023-04-11 03:57:17.754617 | controller | return self._connection_cls(self)
2023-04-11 03:57:17.754628 | controller |
2023-04-11 03:57:17.754639 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 147, in __init__
2023-04-11 03:57:17.754650 | controller | Connection._handle_dbapi_exception_noconnection(
2023-04-11 03:57:17.754660 | controller |
2023-04-11 03:57:17.754671 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2410, in _handle_dbapi_exception_noconnection
2023-04-11 03:57:17.754682 | controller | raise newraise.with_traceback(exc_info[2]) from e
2023-04-11 03:57:17.754693 | controller |
2023-04-11 03:57:17.754704 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
2023-04-11 03:57:17.754715 | controller | self._dbapi_connection = engine.raw_connection()
2023-04-11 03:57:17.754726 | controller |
2023-04-11 03:57:17.754737 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3275, in raw_connection
2023-04-11 03:57:17.754748 | controller | return self.pool.connect()
2023-04-11 03:57:17.754764 | controller |
2023-04-11 03:57:17.754776 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 455, in connect
2023-04-11 03:57:17.754787 | controller | return _ConnectionFairy._checkout(self)
2023-04-11 03:57:17.754798 | controller |
2023-04-11 03:57:17.754809 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1271, in _checkout
2023-04-11 03:57:17.754820 | controller | fairy = _ConnectionRecord.checkout(pool)
2023-04-11 03:57:17.754831 | controller |
2023-04-11 03:57:17.754842 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 719, in checkout
2023-04-11 03:57:17.754853 | controller | rec = pool._do_get()
2023-04-11 03:57:17.754864 | controller |
2023-04-11 03:57:17.754881 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get
2023-04-11 03:57:17.754893 | controller | with util.safe_reraise():
2023-04-11 03:57:17.754905 | controller |
2023-04-11 03:57:17.754916 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
2023-04-11 03:57:17.754926 | controller | raise exc_value.with_traceback(exc_tb)
2023-04-11 03:57:17.754938 | controller |
2023-04-11 03:57:17.754949 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 166, in _do_get
2023-04-11 03:57:17.754960 | controller | return self._create_connection()
2023-04-11 03:57:17.754971 | controller |
2023-04-11 03:57:17.754982 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 396, in _create_connection
2023-04-11 03:57:17.754999 | controller | return _ConnectionRecord(self)
2023-04-11 03:57:17.755011 | controller |
2023-04-11 03:57:17.755022 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 681, in __init__
2023-04-11 03:57:17.755033 | controller | self.__connect()
2023-04-11 03:57:17.755044 | controller |
2023-04-11 03:57:17.755055 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 905, in __connect
2023-04-11 03:57:17.755066 | controller | with util.safe_reraise():
2023-04-11 03:57:17.755076 | controller |
2023-04-11 03:57:17.755091 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
2023-04-11 03:57:17.755103 | controller | raise exc_value.with_traceback(exc_tb)
2023-04-11 03:57:17.755114 | controller |
2023-04-11 03:57:17.755125 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 901, in __connect
2023-04-11 03:57:17.755136 | controller | self.dbapi_connection = connection = pool._invoke_creator(self)
2023-04-11 03:57:17.755147 | controller |
2023-04-11 03:57:17.755158 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 636, in connect
2023-04-11 03:57:17.755169 | controller | return dialect.connect(*cargs, **cparams)
2023-04-11 03:57:17.755180 | controller |
2023-04-11 03:57:17.755191 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 579, in connect
2023-04-11 03:57:17.755202 | controller | return self.loaded_dbapi.connect(*cargs, **cparams)
2023-04-11 03:57:17.755213 | controller |
2023-04-11 03:57:17.755224 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect
2023-04-11 03:57:17.755235 | controller | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
2023-04-11 03:57:17.755246 | controller |
2023-04-11 03:57:17.755257 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/eventlet/support/psycopg2_patcher.py", line 46, in eventlet_wait_callback
2023-04-11 03:57:17.755268 | controller | state = conn.poll()
2023-04-11 03:57:17.755279 | controller |
2023-04-11 03:57:17.755290 | controller | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user "openstack_citest"
2023-04-11 03:57:17.755301 | controller | connection to server at "localhost" (::1), port 5432 failed: FATAL: password authentication failed for user "openstack_citest"
2023-04-11 03:57:17.755311 | controller |
2023-04-11 03:57:17.755322 | controller | (Background on this error at: https://sqlalche.me/e/20/e3q8)
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2015847/+subscriptions
References
