yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #87719
[Bug 1952004] [NEW] ovn: Functional test may fail if northd hasn't initialized DBs yet
Public bug reported:
Example:
https://7bed286b1abc124aef60-716c2febf7f730d66787c22f3ed0da3e.ssl.cf5.rackcdn.com/818067/2/check/neutron-functional-with-uwsgi/48adb0f/testr_results.html
Traceback:
2021-11-19 22:15:09.291 52011 INFO neutron.tests.unit.plugins.ml2.drivers.mechanism_logger [req-819d37a7-3659-4d27-8a49-834bc474728f - tenid - - -] create_network_precommit called with network settings {'id': 'da5ba714-bfd7-45f8-99bd-55ff4bee2f89', 'name': 'n1', 'tenant_id': '46f70361-ba71-4bd0-9769-3573fd227c4b', 'admin_state_up': True, 'mtu': 1500, 'status': 'ACTIVE', 'subnets': [], 'standard_attr_id': 6, 'shared': False, 'project_id': '46f70361-ba71-4bd0-9769-3573fd227c4b', 'port_security_enabled': True, 'router:external': False, 'provider:network_type': 'vlan', 'provider:physical_network': 'physnet1', 'provider:segmentation_id': 100, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'vlan_transparent': None, 'description': '', 'revision_number': 1, 'l2_adjacency': True} (original settings None) and network segments [{'id': 'd146263d-2926-40a8-b21a-a6b9a930fbff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 100, 'network_id': 'da5ba714-bfd7-45f8-99bd-55ff4bee2f89'}]
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers [req-819d37a7-3659-4d27-8a49-834bc474728f - tenid - - -] Mechanism driver 'ovn' failed in create_network_precommit: StopIteration
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers Traceback (most recent call last):
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/managers.py", line 482, in _call_on_drivers
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers getattr(driver.obj, method_name)(context)
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 536, in create_network_precommit
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers self._validate_network_segments(context.network_segments)
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 488, in _validate_network_segments
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers max_tunid = self._get_max_tunid()
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 482, in _get_max_tunid
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers return int(self.nb_ovn.nb_global.options.get('max_tunid'))
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 256, in nb_global
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers return next(iter(self.db_list_rows('NB_Global').execute(
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers StopIteration
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers
2021-11-19 22:15:09.296 52011 DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] Hash Ring: Node e6333d61-3ac1-4c6f-9b90-a23e848ed629 (host: ubuntu-focal-inmotion-iad3-0027436269) handling event "create" for row c5936e2f-4eb2-461c-b095-a27908868816 (table: NB_Global) notify /home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py:671
This can happen in cases when northd is slow or local IDL hasn't got
updates yet when northd populates NB_Global table and at the same time
we try to access any of the content in NB_Global.
** Affects: neutron
Importance: Undecided
Assignee: Jakub Libosvar (libosvar)
Status: In Progress
** Tags: functional-tests ovn
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1952004
Title:
ovn: Functional test may fail if northd hasn't initialized DBs yet
Status in neutron:
In Progress
Bug description:
Example:
https://7bed286b1abc124aef60-716c2febf7f730d66787c22f3ed0da3e.ssl.cf5.rackcdn.com/818067/2/check/neutron-functional-with-uwsgi/48adb0f/testr_results.html
Traceback:
2021-11-19 22:15:09.291 52011 INFO neutron.tests.unit.plugins.ml2.drivers.mechanism_logger [req-819d37a7-3659-4d27-8a49-834bc474728f - tenid - - -] create_network_precommit called with network settings {'id': 'da5ba714-bfd7-45f8-99bd-55ff4bee2f89', 'name': 'n1', 'tenant_id': '46f70361-ba71-4bd0-9769-3573fd227c4b', 'admin_state_up': True, 'mtu': 1500, 'status': 'ACTIVE', 'subnets': [], 'standard_attr_id': 6, 'shared': False, 'project_id': '46f70361-ba71-4bd0-9769-3573fd227c4b', 'port_security_enabled': True, 'router:external': False, 'provider:network_type': 'vlan', 'provider:physical_network': 'physnet1', 'provider:segmentation_id': 100, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'vlan_transparent': None, 'description': '', 'revision_number': 1, 'l2_adjacency': True} (original settings None) and network segments [{'id': 'd146263d-2926-40a8-b21a-a6b9a930fbff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 100, 'network_id': 'da5ba714-bfd7-45f8-99bd-55ff4bee2f89'}]
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers [req-819d37a7-3659-4d27-8a49-834bc474728f - tenid - - -] Mechanism driver 'ovn' failed in create_network_precommit: StopIteration
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers Traceback (most recent call last):
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/managers.py", line 482, in _call_on_drivers
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers getattr(driver.obj, method_name)(context)
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 536, in create_network_precommit
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers self._validate_network_segments(context.network_segments)
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 488, in _validate_network_segments
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers max_tunid = self._get_max_tunid()
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 482, in _get_max_tunid
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers return int(self.nb_ovn.nb_global.options.get('max_tunid'))
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 256, in nb_global
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers return next(iter(self.db_list_rows('NB_Global').execute(
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers StopIteration
2021-11-19 22:15:09.292 52011 ERROR neutron.plugins.ml2.managers
2021-11-19 22:15:09.296 52011 DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] Hash Ring: Node e6333d61-3ac1-4c6f-9b90-a23e848ed629 (host: ubuntu-focal-inmotion-iad3-0027436269) handling event "create" for row c5936e2f-4eb2-461c-b095-a27908868816 (table: NB_Global) notify /home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py:671
This can happen in cases when northd is slow or local IDL hasn't got
updates yet when northd populates NB_Global table and at the same time
we try to access any of the content in NB_Global.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1952004/+subscriptions
Follow ups