yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2065779] [NEW] neutron-fullstack-with-uwsgi job broken at resource setup

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: yatin <2065779@xxxxxxxxxxxxxxxxxx>
Date: Wed, 15 May 2024 12:38:19 -0000
Reply-to: Bug 2065779 <2065779@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

All tests fails as:-
ft1.1: neutron.tests.fullstack.test_agent_bandwidth_report.TestAgentBandwidthReport.test_agent_configurations(Open vSwitch agent)testtools.testresult.real._StringException: pythonlogging:'': {{{ERROR [stevedore.extension] Could not load 'oslo_cache.memcache_pool': No module named 'memcache'}}}
traceback-1: {{{
Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
    eventlet.sleep(sleep)
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
    hub.switch()
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
    return self.greenlet.switch()
eventlet.timeout.Timeout: 60 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
    self._setUp()
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
    common_utils.wait_until_true(self.server_is_live)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
    raise WaitTimeout(_("Timed out after %d seconds") % timeout)
neutron.common.utils.WaitTimeout: Timed out after 60 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 207, in setUp
    raise SetupError(details)
fixtures.fixture.SetupError: {}
}}}

traceback-2: {{{
Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
    eventlet.sleep(sleep)
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
    hub.switch()
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
    return self.greenlet.switch()
eventlet.timeout.Timeout: 60 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
    self._setUp()
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
    common_utils.wait_until_true(self.server_is_live)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
    raise WaitTimeout(_("Timed out after %d seconds") % timeout)
neutron.common.utils.WaitTimeout: Timed out after 60 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
    self._setUp()
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/environment.py", line 423, in _setUp
    self.neutron_server = self.useFixture(
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 256, in useFixture
    fixture.setUp()
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 211, in setUp
    raise MultipleExceptions(*errors)
testtools.runtest.MultipleExceptions: ((<class 'neutron.common.utils.WaitTimeout'>, WaitTimeout('Timed out after 60 seconds'), <traceback object at 0x7f849a0df280>), (<class 'fixtures.fixture.SetupError'>, SetupError({}), <traceback object at 0x7f849a3a9cc0>))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 207, in setUp
    raise SetupError(details)
fixtures.fixture.SetupError: {}
}}}

Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
    eventlet.sleep(sleep)
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
    hub.switch()
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
    return self.greenlet.switch()
eventlet.timeout.Timeout: 60 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
    self._setUp()
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
    common_utils.wait_until_true(self.server_is_live)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
    raise WaitTimeout(_("Timed out after %d seconds") % timeout)
neutron.common.utils.WaitTimeout: Timed out after 60 seconds

neutron server fails as:-
2024-05-14 13:16:19.166 68768 ERROR neutron.service [None req-f40f8906-b901-4249-b984-b87293fbeb92 - - - - - -] Unrecoverable error: please check log for details.: sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'openstack_citest'@'localhost' (using password: YES)")
(Background on this error at: https://sqlalche.me/e/20/e3q8)
2024-05-14 13:16:19.166 68768 ERROR neutron.service Traceback (most recent call last):
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
2024-05-14 13:16:19.166 68768 ERROR neutron.service     self._dbapi_connection = engine.raw_connection()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3304, in raw_connection
2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self.pool.connect()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 449, in connect
2024-05-14 13:16:19.166 68768 ERROR neutron.service     return _ConnectionFairy._checkout(self)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
2024-05-14 13:16:19.166 68768 ERROR neutron.service     fairy = _ConnectionRecord.checkout(pool)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
2024-05-14 13:16:19.166 68768 ERROR neutron.service     rec = pool._do_get()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
2024-05-14 13:16:19.166 68768 ERROR neutron.service     with util.safe_reraise():
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise exc_value.with_traceback(exc_tb)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self._create_connection()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
2024-05-14 13:16:19.166 68768 ERROR neutron.service     return _ConnectionRecord(self)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.__connect()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
2024-05-14 13:16:19.166 68768 ERROR neutron.service     with util.safe_reraise():
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise exc_value.with_traceback(exc_tb)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.dbapi_connection = connection = pool._invoke_creator(self)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 643, in connect
2024-05-14 13:16:19.166 68768 ERROR neutron.service     return dialect.connect(*cargs, **cparams)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 617, in connect
2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self.loaded_dbapi.connect(*cargs, **cparams)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 358, in __init__
2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.connect()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 664, in connect
2024-05-14 13:16:19.166 68768 ERROR neutron.service     self._request_authentication()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 976, in _request_authentication
2024-05-14 13:16:19.166 68768 ERROR neutron.service     auth_packet = _auth.caching_sha2_password_auth(self, auth_packet)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/_auth.py", line 267, in caching_sha2_password_auth
2024-05-14 13:16:19.166 68768 ERROR neutron.service     pkt = _roundtrip(conn, data)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/_auth.py", line 120, in _roundtrip
2024-05-14 13:16:19.166 68768 ERROR neutron.service     pkt = conn._read_packet()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet 
2024-05-14 13:16:19.166 68768 ERROR neutron.service     packet.raise_for_error()
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
2024-05-14 13:16:19.166 68768 ERROR neutron.service     err.raise_mysql_exception(self._data)
2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise errorclass(errno, errval)
2024-05-14 13:16:19.166 68768 ERROR neutron.service pymysql.err.OperationalError: (1045, "Access denied for user 'openstack_citest'@'localhost' (using password: YES)")
2024-05-14 13:16:19.166 68768 ERROR neutron.service     


Example failure:- https://688891352003d7aa1ebf-eb9cf67f730d66b65f59031307d31060.ssl.cf1.rackcdn.com/919602/2/check/neutron-fullstack-with-uwsgi/94787d6/testr_results.html

Builds:- https://zuul.opendev.org/t/openstack/builds?job_name=neutron-
fullstack-with-uwsgi&project=openstack/neutron

** Affects: neutron
     Importance: Critical
         Status: New


** Tags: fullstack gate-failure

** Changed in: neutron
   Importance: Undecided => Critical

** Tags added: fullstack gate-failure

** Description changed:

  All tests fails as:-
  ft1.1: neutron.tests.fullstack.test_agent_bandwidth_report.TestAgentBandwidthReport.test_agent_configurations(Open vSwitch agent)testtools.testresult.real._StringException: pythonlogging:'': {{{ERROR [stevedore.extension] Could not load 'oslo_cache.memcache_pool': No module named 'memcache'}}}
  traceback-1: {{{
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
-     eventlet.sleep(sleep)
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
-     hub.switch()
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
-     return self.greenlet.switch()
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
+     eventlet.sleep(sleep)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
+     hub.switch()
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
+     return self.greenlet.switch()
  eventlet.timeout.Timeout: 60 seconds
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
-     self._setUp()
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
-     common_utils.wait_until_true(self.server_is_live)
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
-     raise WaitTimeout(_("Timed out after %d seconds") % timeout)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
+     self._setUp()
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
+     common_utils.wait_until_true(self.server_is_live)
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
+     raise WaitTimeout(_("Timed out after %d seconds") % timeout)
  neutron.common.utils.WaitTimeout: Timed out after 60 seconds
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 207, in setUp
-     raise SetupError(details)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 207, in setUp
+     raise SetupError(details)
  fixtures.fixture.SetupError: {}
  }}}
  
  traceback-2: {{{
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
-     eventlet.sleep(sleep)
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
-     hub.switch()
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
-     return self.greenlet.switch()
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
+     eventlet.sleep(sleep)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
+     hub.switch()
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
+     return self.greenlet.switch()
  eventlet.timeout.Timeout: 60 seconds
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
-     self._setUp()
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
-     common_utils.wait_until_true(self.server_is_live)
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
-     raise WaitTimeout(_("Timed out after %d seconds") % timeout)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
+     self._setUp()
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
+     common_utils.wait_until_true(self.server_is_live)
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
+     raise WaitTimeout(_("Timed out after %d seconds") % timeout)
  neutron.common.utils.WaitTimeout: Timed out after 60 seconds
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
-     self._setUp()
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/environment.py", line 423, in _setUp
-     self.neutron_server = self.useFixture(
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 256, in useFixture
-     fixture.setUp()
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 211, in setUp
-     raise MultipleExceptions(*errors)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
+     self._setUp()
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/environment.py", line 423, in _setUp
+     self.neutron_server = self.useFixture(
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 256, in useFixture
+     fixture.setUp()
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 211, in setUp
+     raise MultipleExceptions(*errors)
  testtools.runtest.MultipleExceptions: ((<class 'neutron.common.utils.WaitTimeout'>, WaitTimeout('Timed out after 60 seconds'), <traceback object at 0x7f849a0df280>), (<class 'fixtures.fixture.SetupError'>, SetupError({}), <traceback object at 0x7f849a3a9cc0>))
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 207, in setUp
-     raise SetupError(details)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 207, in setUp
+     raise SetupError(details)
  fixtures.fixture.SetupError: {}
  }}}
  
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
-     eventlet.sleep(sleep)
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
-     hub.switch()
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
-     return self.greenlet.switch()
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
+     eventlet.sleep(sleep)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
+     hub.switch()
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
+     return self.greenlet.switch()
  eventlet.timeout.Timeout: 60 seconds
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
-   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
-     self._setUp()
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
-     common_utils.wait_until_true(self.server_is_live)
-   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
-     raise WaitTimeout(_("Timed out after %d seconds") % timeout)
+   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
+     self._setUp()
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
+     common_utils.wait_until_true(self.server_is_live)
+   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
+     raise WaitTimeout(_("Timed out after %d seconds") % timeout)
  neutron.common.utils.WaitTimeout: Timed out after 60 seconds
+ 
+ neutron server fails as:-
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service [None req-f40f8906-b901-4249-b984-b87293fbeb92 - - - - - -] Unrecoverable error: please check log for details.: sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'openstack_citest'@'localhost' (using password: YES)")
+ (Background on this error at: https://sqlalche.me/e/20/e3q8)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service Traceback (most recent call last):
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     self._dbapi_connection = engine.raw_connection()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3304, in raw_connection
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self.pool.connect()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 449, in connect
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     return _ConnectionFairy._checkout(self)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     fairy = _ConnectionRecord.checkout(pool)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     rec = pool._do_get()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     with util.safe_reraise():
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise exc_value.with_traceback(exc_tb)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self._create_connection()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     return _ConnectionRecord(self)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.__connect()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     with util.safe_reraise():
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise exc_value.with_traceback(exc_tb)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.dbapi_connection = connection = pool._invoke_creator(self)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 643, in connect
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     return dialect.connect(*cargs, **cparams)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 617, in connect
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self.loaded_dbapi.connect(*cargs, **cparams)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 358, in __init__
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.connect()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 664, in connect
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     self._request_authentication()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 976, in _request_authentication
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     auth_packet = _auth.caching_sha2_password_auth(self, auth_packet)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/_auth.py", line 267, in caching_sha2_password_auth
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     pkt = _roundtrip(conn, data)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/_auth.py", line 120, in _roundtrip
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     pkt = conn._read_packet()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet 
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     packet.raise_for_error()
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     err.raise_mysql_exception(self._data)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise errorclass(errno, errval)
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service pymysql.err.OperationalError: (1045, "Access denied for user 'openstack_citest'@'localhost' (using password: YES)")
+ 2024-05-14 13:16:19.166 68768 ERROR neutron.service     
  
  
  Example failure:- https://688891352003d7aa1ebf-eb9cf67f730d66b65f59031307d31060.ssl.cf1.rackcdn.com/919602/2/check/neutron-fullstack-with-uwsgi/94787d6/testr_results.html
  
  Builds:- https://zuul.opendev.org/t/openstack/builds?job_name=neutron-
  fullstack-with-uwsgi&project=openstack/neutron

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2065779

Title:
  neutron-fullstack-with-uwsgi job broken at resource setup

Status in neutron:
  New

Bug description:
  All tests fails as:-
  ft1.1: neutron.tests.fullstack.test_agent_bandwidth_report.TestAgentBandwidthReport.test_agent_configurations(Open vSwitch agent)testtools.testresult.real._StringException: pythonlogging:'': {{{ERROR [stevedore.extension] Could not load 'oslo_cache.memcache_pool': No module named 'memcache'}}}
  traceback-1: {{{
  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
      eventlet.sleep(sleep)
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
      hub.switch()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
      return self.greenlet.switch()
  eventlet.timeout.Timeout: 60 seconds

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
      self._setUp()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
      common_utils.wait_until_true(self.server_is_live)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
      raise WaitTimeout(_("Timed out after %d seconds") % timeout)
  neutron.common.utils.WaitTimeout: Timed out after 60 seconds

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 207, in setUp
      raise SetupError(details)
  fixtures.fixture.SetupError: {}
  }}}

  traceback-2: {{{
  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
      eventlet.sleep(sleep)
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
      hub.switch()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
      return self.greenlet.switch()
  eventlet.timeout.Timeout: 60 seconds

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
      self._setUp()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
      common_utils.wait_until_true(self.server_is_live)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
      raise WaitTimeout(_("Timed out after %d seconds") % timeout)
  neutron.common.utils.WaitTimeout: Timed out after 60 seconds

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
      self._setUp()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/environment.py", line 423, in _setUp
      self.neutron_server = self.useFixture(
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 256, in useFixture
      fixture.setUp()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 211, in setUp
      raise MultipleExceptions(*errors)
  testtools.runtest.MultipleExceptions: ((<class 'neutron.common.utils.WaitTimeout'>, WaitTimeout('Timed out after 60 seconds'), <traceback object at 0x7f849a0df280>), (<class 'fixtures.fixture.SetupError'>, SetupError({}), <traceback object at 0x7f849a3a9cc0>))

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 207, in setUp
      raise SetupError(details)
  fixtures.fixture.SetupError: {}
  }}}

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
      eventlet.sleep(sleep)
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 38, in sleep
      hub.switch()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch
      return self.greenlet.switch()
  eventlet.timeout.Timeout: 60 seconds

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/fixtures/fixture.py", line 196, in setUp
      self._setUp()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/process.py", line 175, in _setUp
      common_utils.wait_until_true(self.server_is_live)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 748, in wait_until_true
      raise WaitTimeout(_("Timed out after %d seconds") % timeout)
  neutron.common.utils.WaitTimeout: Timed out after 60 seconds

  neutron server fails as:-
  2024-05-14 13:16:19.166 68768 ERROR neutron.service [None req-f40f8906-b901-4249-b984-b87293fbeb92 - - - - - -] Unrecoverable error: please check log for details.: sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'openstack_citest'@'localhost' (using password: YES)")
  (Background on this error at: https://sqlalche.me/e/20/e3q8)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service Traceback (most recent call last):
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     self._dbapi_connection = engine.raw_connection()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3304, in raw_connection
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self.pool.connect()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 449, in connect
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     return _ConnectionFairy._checkout(self)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     fairy = _ConnectionRecord.checkout(pool)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     rec = pool._do_get()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     with util.safe_reraise():
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise exc_value.with_traceback(exc_tb)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self._create_connection()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     return _ConnectionRecord(self)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.__connect()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     with util.safe_reraise():
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise exc_value.with_traceback(exc_tb)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.dbapi_connection = connection = pool._invoke_creator(self)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 643, in connect
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     return dialect.connect(*cargs, **cparams)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 617, in connect
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     return self.loaded_dbapi.connect(*cargs, **cparams)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 358, in __init__
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     self.connect()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 664, in connect
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     self._request_authentication()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 976, in _request_authentication
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     auth_packet = _auth.caching_sha2_password_auth(self, auth_packet)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/_auth.py", line 267, in caching_sha2_password_auth
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     pkt = _roundtrip(conn, data)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/_auth.py", line 120, in _roundtrip
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     pkt = conn._read_packet()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet 
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     packet.raise_for_error()
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     err.raise_mysql_exception(self._data)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     raise errorclass(errno, errval)
  2024-05-14 13:16:19.166 68768 ERROR neutron.service pymysql.err.OperationalError: (1045, "Access denied for user 'openstack_citest'@'localhost' (using password: YES)")
  2024-05-14 13:16:19.166 68768 ERROR neutron.service     

  
  Example failure:- https://688891352003d7aa1ebf-eb9cf67f730d66b65f59031307d31060.ssl.cf1.rackcdn.com/919602/2/check/neutron-fullstack-with-uwsgi/94787d6/testr_results.html

  Builds:- https://zuul.opendev.org/t/openstack/builds?job_name=neutron-
  fullstack-with-uwsgi&project=openstack/neutron

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2065779/+subscriptions
Follow ups

[Bug 2065779] Re: neutron-fullstack-with-uwsgi job broken at resource setup
From: OpenStack Infra, 2024-05-15