yahoo-eng-team team mailing list archive

[Ihar Hrachyshka <2095185@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

This happened in 2023.2 but I suspect it's applicable for master too,
see explanation below.

https://db62f022a051878f64d1-33cd363c3f72485dda255154bdda0fc8.ssl.cf2.rackcdn.com/939237/1/check/neutron-
functional-with-uwsgi/cdc4fc3/testr_results.html

ft1.1: neutron.tests.functional.agent.l3.extensions.test_conntrack_helper_extension.TestL3AgentConntrackHelperExtension.test_legacy_router_conntrack_helpertesttools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 178, in func
    return f(self, *args, **kwargs)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/extensions/test_conntrack_helper_extension.py", line 121, in test_legacy_router_conntrack_helper
    self._test_centralized_routers(router_info)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/extensions/test_conntrack_helper_extension.py", line 110, in _test_centralized_routers
    ri = self.manage_router(self.agent, router_info)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/framework.py", line 413, in manage_router
    agent._process_added_router(router)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/agent.py", line 651, in _process_added_router
    with excutils.save_and_reraise_exception():
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/agent.py", line 649, in _process_added_router
    ri.process()
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 184, in call
    with excutils.save_and_reraise_exception():
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 182, in call
    return func(*args, **kwargs)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 1304, in process
    self._process_internal_ports()
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 690, in _process_internal_ports
    self.internal_network_added(p)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 572, in internal_network_added
    self._internal_network_added(self.ns_name,
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 552, in _internal_network_added
    self.driver.plug(network_id, port_id, interface_name, mac_address,
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/interface.py", line 268, in plug
    self.plug_new(network_id, port_id, device_name, mac_address,
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/interface.py", line 396, in plug_new
    self.check_bridge_exists(bridge)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/interface.py", line 238, in check_bridge_exists
    raise exceptions.BridgeDoesNotExist(bridge=bridge)
neutron_lib.exceptions.BridgeDoesNotExist: Bridge test-bre0e2da16 does not exist.

--

As far as I understand what the test case does is: it sets l3 agent to
use br-test* for integration_bridge. It starts the agent report-state
loop. It then tries to manage a router with the agent. The agent fails
because the integration bridge is not configured on host (no interface
with the name).

I think the test incorrectly assumes that when OVSBridgeFixture created
a bridge in OVSDB, then the kernel interface will immediately pop up.
For this to happen, ovsdb-server should first communicate the change to
vswitchd, then vswitchd should act on this by requesting creating the
bridge interface in kernel.

I think the fix here should be to make the test case (and probably all
test cases that inherit from L3 framework base class
L3AgentTestFramework) to wait until integration bridge is configured,
before returning control to the test case itself. This should be done
somewhere in _configure_agent after 'br_int =
self.useFixture(net_helpers.OVSBridgeFixture()).bridge' line.

** Affects: neutron
     Importance: Undecided
         Status: New


** Tags: functional-tests gate-failure

** Tags added: functional-tests gate-failure

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2095185

Title:
  test_legacy_router_conntrack_helper failed as
  "neutron_lib.exceptions.BridgeDoesNotExist: Bridge test-bre0e2da16
  does not exist" when router attempted to plug interface

Status in neutron:
  New

Bug description:
  This happened in 2023.2 but I suspect it's applicable for master too,
  see explanation below.

  https://db62f022a051878f64d1-33cd363c3f72485dda255154bdda0fc8.ssl.cf2.rackcdn.com/939237/1/check/neutron-
  functional-with-uwsgi/cdc4fc3/testr_results.html

  ft1.1: neutron.tests.functional.agent.l3.extensions.test_conntrack_helper_extension.TestL3AgentConntrackHelperExtension.test_legacy_router_conntrack_helpertesttools.testresult.real._StringException: Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 178, in func
      return f(self, *args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/extensions/test_conntrack_helper_extension.py", line 121, in test_legacy_router_conntrack_helper
      self._test_centralized_routers(router_info)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/extensions/test_conntrack_helper_extension.py", line 110, in _test_centralized_routers
      ri = self.manage_router(self.agent, router_info)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/framework.py", line 413, in manage_router
      agent._process_added_router(router)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/agent.py", line 651, in _process_added_router
      with excutils.save_and_reraise_exception():
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
      self.force_reraise()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
      raise self.value
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/agent.py", line 649, in _process_added_router
      ri.process()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 184, in call
      with excutils.save_and_reraise_exception():
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
      self.force_reraise()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
      raise self.value
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 182, in call
      return func(*args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 1304, in process
      self._process_internal_ports()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 690, in _process_internal_ports
      self.internal_network_added(p)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 572, in internal_network_added
      self._internal_network_added(self.ns_name,
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 552, in _internal_network_added
      self.driver.plug(network_id, port_id, interface_name, mac_address,
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/interface.py", line 268, in plug
      self.plug_new(network_id, port_id, device_name, mac_address,
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/interface.py", line 396, in plug_new
      self.check_bridge_exists(bridge)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/interface.py", line 238, in check_bridge_exists
      raise exceptions.BridgeDoesNotExist(bridge=bridge)
  neutron_lib.exceptions.BridgeDoesNotExist: Bridge test-bre0e2da16 does not exist.

  --

  As far as I understand what the test case does is: it sets l3 agent to
  use br-test* for integration_bridge. It starts the agent report-state
  loop. It then tries to manage a router with the agent. The agent fails
  because the integration bridge is not configured on host (no interface
  with the name).

  I think the test incorrectly assumes that when OVSBridgeFixture
  created a bridge in OVSDB, then the kernel interface will immediately
  pop up. For this to happen, ovsdb-server should first communicate the
  change to vswitchd, then vswitchd should act on this by requesting
  creating the bridge interface in kernel.

  I think the fix here should be to make the test case (and probably all
  test cases that inherit from L3 framework base class
  L3AgentTestFramework) to wait until integration bridge is configured,
  before returning control to the test case itself. This should be done
  somewhere in _configure_agent after 'br_int =
  self.useFixture(net_helpers.OVSBridgeFixture()).bridge' line.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2095185/+subscriptions