yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1903008] Re: Create network failed during functional test

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Rodolfo Alonso <1903008@xxxxxxxxxxxxxxxxxx>
Date: Wed, 30 Nov 2022 09:01:18 -0000
Reply-to: Bug 1903008 <1903008@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
** Changed in: neutron
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1903008

Title:
  Create network failed during functional test

Status in neutron:
  Fix Released

Bug description:
  One of the functional, ovn related tests failed with error like:

  ft1.3: neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.ovsdb.test_ovsdb_monitor.TestNBDbMonitorOverTcp.test_distributed_locktesttools.testresult.real._StringException: Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
      return f(self, *args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
      return f(self, *args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/plugins/ml2/drivers/ovn/mech_driver/ovsdb/test_ovsdb_monitor.py", line 231, in test_distributed_lock
      self.create_port()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/plugins/ml2/drivers/ovn/mech_driver/ovsdb/test_ovsdb_monitor.py", line 77, in create_port
      net = self._make_network(self.fmt, 'net1', True)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/unit/db/test_db_base_plugin_v2.py", line 510, in _make_network
      raise webob.exc.HTTPClientError(code=res.status_int)
  webob.exc.HTTPClientError: The server could not comply with the request since it is either malformed or otherwise incorrect.

  
  In the testrun log there is error like:
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers Traceback (most recent call last):
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/ovn_revision_numbers_db.py", line 93, in _get_standard_attr_id
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers     row = context.session.query(STD_ATTR_MAP[resource_type]).filter_by(
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3500, in one
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers     raise orm_exc.NoResultFound("No row was found for one()")
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers sqlalchemy.orm.exc.NoResultFound: No row was found for one()
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers 
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers During handling of the above exception, another exception occurred:
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers 
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers Traceback (most recent call last):
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/managers.py", line 477, in _call_on_drivers
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers     getattr(driver.obj, method_name)(context)
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 471, in create_network_precommit
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers     ovn_revision_numbers_db.create_initial_revision(
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 233, in wrapped
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers     return method(*args, **kwargs)
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/ovn_revision_numbers_db.py", line 108, in create_initial_revision
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers     std_attr_id = _get_standard_attr_id(
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/ovn_revision_numbers_db.py", line 97, in _get_standard_attr_id
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers     raise StandardAttributeIDNotFound(resource_uuid=resource_uuid)
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers neutron.db.ovn_revision_numbers_db.StandardAttributeIDNotFound: Standard attribute ID not found for 3f787ea1-0ce5-4310-9b42-b530e7dcb4a1
  2020-11-04 13:34:50.442 61672 ERROR neutron.plugins.ml2.managers 
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation [req-5a27b67c-a80f-4877-84cb-7b3d511d51d6 - tenid - - -] POST failed.: neutron.plugins.ml2.common.exceptions.MechanismDriverError
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last):
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/pecan/core.py", line 683, in __call__
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self.invoke_controller(controller, args, kwargs, state)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/pecan/core.py", line 574, in invoke_controller
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     result = controller(*args, **kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 139, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     setattr(e, '_RETRY_EXCEEDED', True)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/six.py", line 703, in reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     raise value
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 135, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_db/api.py", line 154, in wrapper
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     ectxt.value = e.inner_exc
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/six.py", line 703, in reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     raise value
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_db/api.py", line 142, in wrapper
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 183, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     LOG.debug("Retry wrapper got retriable exception: %s", e)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/six.py", line 703, in reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     raise value
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 179, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return f(*dup_args, **dup_kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/pecan_wsgi/controllers/utils.py", line 76, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/pecan_wsgi/controllers/resource.py", line 163, in post
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return self.create(resources)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/pecan_wsgi/controllers/resource.py", line 181, in create
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return {key: creator(*creator_args, **creator_kwargs)}
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 687, in inner
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return f(self, context, *args, **kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 233, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return method(*args, **kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 139, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     setattr(e, '_RETRY_EXCEEDED', True)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/six.py", line 703, in reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     raise value
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 135, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_db/api.py", line 154, in wrapper
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     ectxt.value = e.inner_exc
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/six.py", line 703, in reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     raise value
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_db/api.py", line 142, in wrapper
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 183, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     LOG.debug("Retry wrapper got retriable exception: %s", e)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/six.py", line 703, in reraise
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     raise value
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/neutron_lib/db/api.py", line 179, in wrapped
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     return f(*dup_args, **dup_kwargs)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/plugin.py", line 1068, in create_network
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     result, mech_context = self._create_network_db(context, network)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/plugin.py", line 1061, in _create_network_db
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self.mechanism_manager.create_network_precommit(mech_context)
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/managers.py", line 512, in create_network_precommit
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     self._call_on_drivers("create_network_precommit", context,
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation   File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/managers.py", line 493, in _call_on_drivers
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation     raise ml2_exc.MechanismDriverError(
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation neutron.plugins.ml2.common.exceptions.MechanismDriverError
  2020-11-04 13:34:50.448 61672 ERROR neutron.pecan_wsgi.hooks.translation 

  Link to error logs:
  https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_236/759673/5/check/neutron-
  functional-with-uwsgi/236ded2/testr_results.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1903008/+subscriptions
References

[Bug 1903008] [NEW] Create network failed during functional test
From: Slawek Kaplonski, 2020-11-05