yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2115629] Re: mariadb and centos 9 stream mysql tempest and functiona jobs failing with sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1273, "Unknown collation: 'utf8mb4_0900_as_cs'")

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: OpenStack Infra <2115629@xxxxxxxxxxxxxxxxxx>
Date: Tue, 01 Jul 2025 00:25:48 -0000
Reply-to: Bug 2115629 <2115629@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Reviewed:  https://review.opendev.org/c/openstack/neutron/+/953768
Committed: https://opendev.org/openstack/neutron/commit/e5606b776eb3ee591ffda7504b7059be71adc23b
Submitter: "Zuul (22348)"
Branch:    master

commit e5606b776eb3ee591ffda7504b7059be71adc23b
Author: Rodolfo Alonso Hernandez <ralonsoh@xxxxxxxxxx>
Date:   Mon Jun 30 13:33:42 2025 +0000

    Change "tag" table collation to "utf8mb4_bin"
    
    The collation "utf8mb4_bin" is case sensitive and is present in:
    * MariaDB: all released versions.
    * MySQL: since 5.5.3 (2010).
    
    Closes-Bug: #2115629
    Related-Bug: #2114819
    Change-Id: Ice1cf5f03437a470b4732a009a4cf60471ab0021


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2115629

Title:
  mariadb and centos 9 stream mysql tempest and functiona jobs failing
  with sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError)
  (1273, "Unknown collation: 'utf8mb4_0900_as_cs'")

Status in neutron:
  Fix Released

Bug description:
  Fails as:- 2025-06-30 02:58:22.687097 | controller | INFO  [alembic.runtime.migration] Running upgrade ad80a9f07c5c -> d553edeb540f
  2025-06-30 02:58:22.687115 | controller | Running upgrade for neutron ...
  2025-06-30 02:58:22.687137 | controller | Traceback (most recent call last):
  2025-06-30 02:58:22.687153 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/base.py", line 1963, in _exec_single_context
  2025-06-30 02:58:22.687187 | controller |     self.dialect.do_execute(
  2025-06-30 02:58:22.687211 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute
  2025-06-30 02:58:22.687241 | controller |     cursor.execute(statement, parameters)
  2025-06-30 02:58:22.687262 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/cursors.py", line 153, in execute
  2025-06-30 02:58:22.687280 | controller |     result = self._query(query)
  2025-06-30 02:58:22.687312 | controller |              ^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.687341 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/cursors.py", line 322, in _query
  2025-06-30 02:58:22.687362 | controller |     conn.query(q)
  2025-06-30 02:58:22.687382 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/connections.py", line 563, in query
  2025-06-30 02:58:22.687401 | controller |     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2025-06-30 02:58:22.687420 | controller |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.687439 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/connections.py", line 825, in _read_query_result
  2025-06-30 02:58:22.687462 | controller |     result.read()
  2025-06-30 02:58:22.687484 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/connections.py", line 1199, in read
  2025-06-30 02:58:22.687526 | controller |     first_packet = self.connection._read_packet()
  2025-06-30 02:58:22.687541 | controller |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.687558 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/connections.py", line 775, in _read_packet
  2025-06-30 02:58:22.687575 | controller |     packet.raise_for_error()
  2025-06-30 02:58:22.687591 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/protocol.py", line 219, in raise_for_error
  2025-06-30 02:58:22.687764 | controller |     err.raise_mysql_exception(self._data)
  2025-06-30 02:58:22.688108 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
  2025-06-30 02:58:22.688132 | controller |     raise errorclass(errno, errval)
  2025-06-30 02:58:22.688147 | controller | pymysql.err.OperationalError: (1273, "Unknown collation: 'utf8mb4_0900_as_cs'")
  2025-06-30 02:58:22.688161 | controller |
  2025-06-30 02:58:22.688211 | controller | The above exception was the direct cause of the following exception:
  2025-06-30 02:58:22.688230 | controller |
  2025-06-30 02:58:22.688245 | controller | Traceback (most recent call last):
  2025-06-30 02:58:22.688260 | controller |   File "/usr/local/bin/neutron-db-manage", line 8, in <module>
  2025-06-30 02:58:22.688288 | controller |     sys.exit(main())
  2025-06-30 02:58:22.688307 | controller |              ^^^^^^
  2025-06-30 02:58:22.688324 | controller |   File "/opt/stack/neutron/neutron/db/migration/cli.py", line 670, in main
  2025-06-30 02:58:22.688342 | controller |     return_val |= bool(CONF.command.func(config, CONF.command.name))
  2025-06-30 02:58:22.688358 | controller |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.688372 | controller |   File "/opt/stack/neutron/neutron/db/migration/cli.py", line 192, in do_upgrade
  2025-06-30 02:58:22.688386 | controller |     do_alembic_command(config, cmd, revision=revision,
  2025-06-30 02:58:22.688400 | controller |   File "/opt/stack/neutron/neutron/db/migration/cli.py", line 94, in do_alembic_command
  2025-06-30 02:58:22.688414 | controller |     getattr(alembic_command, cmd)(config, *args, **kwargs)
  2025-06-30 02:58:22.688428 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/command.py", line 483, in upgrade
  2025-06-30 02:58:22.688442 | controller |     script.run_env()
  2025-06-30 02:58:22.688456 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/script/base.py", line 551, in run_env
  2025-06-30 02:58:22.688506 | controller |     util.load_python_file(self.dir, "env.py")
  2025-06-30 02:58:22.688522 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 116, in load_python_file
  2025-06-30 02:58:22.688536 | controller |     module = load_module_py(module_id, path)
  2025-06-30 02:58:22.688550 | controller |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.688564 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 136, in load_module_py
  2025-06-30 02:58:22.688579 | controller |     spec.loader.exec_module(module)  # type: ignore
  2025-06-30 02:58:22.688623 | controller |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.688640 | controller |   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  2025-06-30 02:58:22.688654 | controller |   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  2025-06-30 02:58:22.688728 | controller |   File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 119, in <module>
  2025-06-30 02:58:22.688751 | controller |     run_migrations_online()
  2025-06-30 02:58:22.688769 | controller |   File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 113, in run_migrations_online
  2025-06-30 02:58:22.688788 | controller |     context.run_migrations()
  2025-06-30 02:58:22.688807 | controller |   File "<string>", line 8, in run_migrations
  2025-06-30 02:58:22.688825 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/runtime/environment.py", line 946, in run_migrations
  2025-06-30 02:58:22.688841 | controller |     self.get_context().run_migrations(**kw)
  2025-06-30 02:58:22.688859 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
  2025-06-30 02:58:22.688875 | controller |     step.migration_fn(**kw)
  2025-06-30 02:58:22.688889 | controller |   File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/versions/2025.2/expand/d553edeb540f_tag_case_sensitive.py", line 31, in upgrade
  2025-06-30 02:58:22.688903 | controller |     op.execute('ALTER TABLE tags CONVERT TO CHARACTER SET utf8mb4 '
  2025-06-30 02:58:22.688917 | controller |   File "<string>", line 8, in execute
  2025-06-30 02:58:22.688931 | controller |   File "<string>", line 3, in execute
  2025-06-30 02:58:22.688945 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/operations/ops.py", line 2591, in execute
  2025-06-30 02:58:22.688960 | controller |     return operations.invoke(op)
  2025-06-30 02:58:22.688984 | controller |            ^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689004 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/operations/base.py", line 441, in invoke
  2025-06-30 02:58:22.689029 | controller |     return fn(self, operation)
  2025-06-30 02:58:22.689043 | controller |            ^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689058 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/operations/toimpl.py", line 240, in execute_sql
  2025-06-30 02:58:22.689072 | controller |     operations.migration_context.impl.execute(
  2025-06-30 02:58:22.689086 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/ddl/impl.py", line 253, in execute
  2025-06-30 02:58:22.689101 | controller |     self._exec(sql, execution_options)
  2025-06-30 02:58:22.689133 | controller |   File "/usr/local/lib/python3.12/site-packages/alembic/ddl/impl.py", line 246, in _exec
  2025-06-30 02:58:22.689151 | controller |     return conn.execute(construct, params)
  2025-06-30 02:58:22.689165 | controller |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689180 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/base.py", line 1415, in execute
  2025-06-30 02:58:22.689228 | controller |     return meth(
  2025-06-30 02:58:22.689249 | controller |            ^^^^^
  2025-06-30 02:58:22.689266 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/sql/elements.py", line 523, in _execute_on_connection
  2025-06-30 02:58:22.689281 | controller |     return connection._execute_clauseelement(
  2025-06-30 02:58:22.689295 | controller |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689309 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/base.py", line 1637, in _execute_clauseelement
  2025-06-30 02:58:22.689330 | controller |     ret = self._execute_context(
  2025-06-30 02:58:22.689345 | controller |           ^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689359 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context
  2025-06-30 02:58:22.689373 | controller |     return self._exec_single_context(
  2025-06-30 02:58:22.689387 | controller |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689401 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/base.py", line 1982, in _exec_single_context
  2025-06-30 02:58:22.689415 | controller |     self._handle_dbapi_exception(
  2025-06-30 02:58:22.689429 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/base.py", line 2351, in _handle_dbapi_exception
  2025-06-30 02:58:22.689443 | controller |     raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  2025-06-30 02:58:22.689458 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/base.py", line 1963, in _exec_single_context
  2025-06-30 02:58:22.689472 | controller |     self.dialect.do_execute(
  2025-06-30 02:58:22.689487 | controller |   File "/usr/local/lib64/python3.12/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute
  2025-06-30 02:58:22.689501 | controller |     cursor.execute(statement, parameters)
  2025-06-30 02:58:22.689515 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/cursors.py", line 153, in execute
  2025-06-30 02:58:22.689534 | controller |     result = self._query(query)
  2025-06-30 02:58:22.689549 | controller |              ^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689564 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/cursors.py", line 322, in _query
  2025-06-30 02:58:22.689578 | controller |     conn.query(q)
  2025-06-30 02:58:22.689593 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/connections.py", line 563, in query
  2025-06-30 02:58:22.689608 | controller |     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2025-06-30 02:58:22.689623 | controller |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689639 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/connections.py", line 825, in _read_query_result
  2025-06-30 02:58:22.689657 | controller |     result.read()
  2025-06-30 02:58:22.689675 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/connections.py", line 1199, in read
  2025-06-30 02:58:22.689699 | controller |     first_packet = self.connection._read_packet()
  2025-06-30 02:58:22.689721 | controller |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-06-30 02:58:22.689739 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/connections.py", line 775, in _read_packet
  2025-06-30 02:58:22.689756 | controller |     packet.raise_for_error()
  2025-06-30 02:58:22.689773 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/protocol.py", line 219, in raise_for_error
  2025-06-30 02:58:22.689789 | controller |     err.raise_mysql_exception(self._data)
  2025-06-30 02:58:22.689803 | controller |   File "/usr/local/lib/python3.12/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
  2025-06-30 02:58:22.689817 | controller |     raise errorclass(errno, errval)
  2025-06-30 02:58:22.689831 | controller | sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1273, "Unknown collation: 'utf8mb4_0900_as_cs'")
  2025-06-30 02:58:22.689872 | controller | [SQL: ALTER TABLE tags CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs]
  2025-06-30 02:58:22.689888 | controller | (Background on this error at: https://sqlalche.me/e/20/e3q8)

  Example failures:-
  https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_5af/openstack/5afd9a3dd76147b896d56946a8fdeb55/job-output.txt
  https://63b837b4ad9a4e528fa6-44886a430c48ba6939226a5d4d9ebe5b.ssl.cf5.rackcdn.com/openstack/e7e27ee524004363a5c9a998b9d15a3c/job-output.txt
  https://4833eea8cb4760a801b6-7a3e129cf8f9aa6f4b3648598f45d86a.ssl.cf5.rackcdn.com/openstack/71baf61777b844578daad15d564ecb23/job-output.txt

  Builds:- https://zuul.openstack.org/builds?job_name=neutron-ovn-tempest-mariadb-full&job_name=neutron-ovn-tempest-ovs-master-centos-9-stream&job_name=neutron-ovn-tempest-ovs-release-fips&job_name=neutron-ovs-tempest-fips&job_name=neutron-functional-fips&job_name=horizon-integration-pytest&branch=master&skip=0
  https://7598b949b48f30e69b3e-9945c6e94a1b9feee2753a6244e7fd57.ssl.cf5.rackcdn.com/openstack/7088e84f4d4347bdb52f7b7432391c40/testr_results.html

  Triggered by https://review.opendev.org/c/openstack/neutron/+/952819

  Also seen in some centos 10-stream jobs
  https://issues.redhat.com/browse/OSPCIX-934

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2115629/+subscriptions
References

[Bug 2115629] [NEW] mariadb and centos 9 stream mysql tempest and functiona jobs failing with sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1273, "Unknown collation: 'utf8mb4_0900_as_cs'")
From: yatin, 2025-06-30