yahoo-eng-team team mailing list archive

[OpenStack Infra <1961740@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/830622
Committed: https://opendev.org/openstack/neutron/commit/f7fac03ae1a4b1b1471f6054ba058382a1b9e1a1
Submitter: "Zuul (22348)"
Branch:    master

commit f7fac03ae1a4b1b1471f6054ba058382a1b9e1a1
Author: Slawek Kaplonski <skaplons@xxxxxxxxxx>
Date:   Wed Feb 23 15:12:37 2022 +0100

    Repeat few times put new interface in the namespace
    
    It seems that from time to time we may experience another variant of the
    shy ovs ports issue (see [1] for details) while trying to put interface,
    e.g. router's port into the namespace by the interface driver.
    To avoid that, this patch adds some repeats of the putting interface in
    the namespace, in the same way like it was done originally with
    set_address of the device.
    
    Additionally, this patch also refactors a bit part which is responsible
    to set mac address of the device to be able to clean ports in case of
    the permanent error there.
    
    [1] https://bugs.launchpad.net/neutron/+bug/1618987
    
    Closes-Bug: #1961740
    Change-Id: I3e0786fd8d0320036b9046746ae579c8ed2ecf27


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1961740

Title:
  Network interface not found in namespace

Status in neutron:
  Fix Released

Bug description:
  This is intermittent error in the functional tests job. Failure
  example:
  https://89b1b88fa362b409cfb1-2a70ac574f4ba34d12afc72df211f1b3.ssl.cf5.rackcdn.com/828687/1/gate/neutron-
  functional-with-uwsgi/a5b844b/testr_results.html

  Stacktrace:
  ft1.47: neutron.tests.functional.agent.l3.extensions.test_port_forwarding_extension.TestL3AgentFipPortForwardingExtensionDVR.test_dvr_non_ha_router_updatetesttools.testresult.real._StringException: Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 183, in func
      return f(self, *args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 183, in func
      return f(self, *args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/test_dvr_router.py", line 1708, in test_dvr_non_ha_router_update
      router2 = self._create_dvr_ha_router(
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/test_dvr_router.py", line 1445, in _create_dvr_ha_router
      router = self.manage_router(agent, r_info)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/test_dvr_router.py", line 195, in manage_router
      return super(TestDvrRouter, self).manage_router(agent, router)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/framework.py", line 420, in manage_router
      agent._process_added_router(router)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/agent.py", line 663, in _process_added_router
      self._cleanup_failed_router(router['id'],
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__
      self.force_reraise()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
      raise self.value
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/agent.py", line 653, in _process_added_router
      ri.process()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/dvr_local_router.py", line 924, in process
      super(DvrLocalRouter, self).process()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/dvr_router_base.py", line 29, in process
      super(DvrRouterBase, self).process()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 177, in call
      self.logger(e)
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__
      self.force_reraise()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
      raise self.value
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 174, in call
      return func(*args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 1312, in process
      self.process_external()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/dvr_local_router.py", line 740, in process_external
      super(DvrLocalRouter, self).process_external()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/decorator.py", line 232, in fun
      return caller(func, *(extras + args), **kw)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/coordination.py", line 78, in _synchronized
      return f(*a, **k)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 1046, in process_external
      self._process_external_gateway(ex_gw_port)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 927, in _process_external_gateway
      self.external_gateway_added(ex_gw_port, interface_name)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/dvr_edge_router.py", line 43, in external_gateway_added
      self._create_dvr_gateway(ex_gw_port, interface_name)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/dvr_edge_router.py", line 194, in _create_dvr_gateway
      self._external_gateway_added(ex_gw_port, gw_interface_name,
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 821, in _external_gateway_added
      self._plug_external_gateway(ex_gw_port, interface_name, ns_name)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l3/router_info.py", line 755, in _plug_external_gateway
      self.driver.plug(ex_gw_port['network_id'],
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/interface.py", line 263, in plug
      self.plug_new(network_id, port_id, device_name, mac_address,
    File "<string>", line 3, in plug_new
    File "/usr/lib/python3.8/unittest/mock.py", line 1081, in __call__
      return self._mock_call(*args, **kwargs)
    File "/usr/lib/python3.8/unittest/mock.py", line 1085, in _mock_call
      return self._execute_mock_call(*args, **kwargs)
    File "/usr/lib/python3.8/unittest/mock.py", line 1146, in _execute_mock_call
      result = effect(*args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/framework.py", line 408, in new_ovs_plug
      original_plug_new(self, *args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/interface.py", line 388, in plug_new
      namespace_obj.add_device_to_namespace(ns_dev)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/ip_lib.py", line 271, in add_device_to_namespace
      device.link.set_netns(self.namespace)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/ip_lib.py", line 462, in set_netns
      privileged.set_link_attribute(
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_privsep/priv_context.py", line 271, in _wrap
      return self.channel.remote_call(name, args, kwargs,
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/oslo_privsep/daemon.py", line 215, in remote_call
      raise exc_type(*result[2])
  neutron.privileged.agent.linux.ip_lib.NetworkInterfaceNotFound: Network interface qg-d9d41d8e-8b not found in namespace None.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1961740/+subscriptions