yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2107775] [NEW] test_assert_pings_during_br_phys_setup_not_lost_in_vlan_to_flat functional tests randomly fails

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: yatin <2107775@xxxxxxxxxxxxxxxxxx>
Date: Mon, 21 Apr 2025 13:26:01 -0000
Reply-to: Bug 2107775 <2107775@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Seen 4 failures in last 8 days, 3 in master, 1 in stable/2023.2
Fails as:-
ft1.18: neutron.tests.functional.agent.test_l2_ovs_agent.TestOVSAgent.test_assert_pings_during_br_phys_setup_not_lost_in_vlan_to_flattesttools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 178, in func
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/test_l2_ovs_agent.py", line 270, in test_assert_pings_during_br_phys_setup_not_lost_in_vlan_to_flat
    self._test_assert_pings_during_br_phys_setup_not_lost(provider_net)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/test_l2_ovs_agent.py", line 305, in _test_assert_pings_during_br_phys_setup_not_lost
    with net_helpers.async_ping(ns_phys, [ip_int]) as done:
  File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
    next(self.gen)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/net_helpers.py", line 150, in async_ping
    f.result()
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/net_helpers.py", line 131, in assert_async_ping
    ns_ip_wrapper.netns.execute(
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/ip_lib.py", line 776, in execute
    return utils.execute(cmd, check_exit_code=check_exit_code,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py", line 157, in execute
    raise exceptions.ProcessExecutionError(msg,
neutron_lib.exceptions.ProcessExecutionError: Exit code: 1; Cmd: ['ip', 'netns', 'exec', 'test-0245f548-3f29-48f0-8d28-0cfe7e68dcc9', 'ping', '-W', 1, '-c', '1', '10.148.117.171']; Stdin: ; Stdout: PING 10.148.117.171 (10.148.117.171) 56(84) bytes of data.

--- 10.148.117.171 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

; Stderr:


Example failures:-
https://03c08251732a708a286f-1284cae0458f2bcc716f47c02bf394ec.ssl.cf2.rackcdn.com/openstack/ffb419b2456b44899920be36e9599cc5/testr_results.html
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a5c/openstack/a5ca7963eaf34ac3846c631e3b52c272/testr_results.html
https://d20c26a0475556dff012-de0170061a4c33caf35d62b404ee94ad.ssl.cf1.rackcdn.com/openstack/d17a470830d5471f8d6f309742a645fe/testr_results.html
https://b19b5ab35f9ced465dd5-2eb50734132c0e56282483bcdf57bf8a.ssl.cf5.rackcdn.com/openstack/8a6caec32ede46adbdcc6aa3b9e8e1c1/testr_results.html

Opensearch(creds: openstack/openstack):-
https://opensearch.logs.openstack.org/_dashboards/app/data-explorer/discover?security_tenant=global#?_a=(discover:(columns:!(_source),isDirty:!f,sort:!()),metadata:(indexPattern:'94869730-aea8-11ec-9e6a-83741af3fdcd',view:discover))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-30d,to:now))&_q=(filters:!(),query:(language:kuery,query:'message:%22in%20test_assert_pings_during_br_phys_setup_not_lost_in_vlan_to_flat%22'))

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2107775

Title:
  test_assert_pings_during_br_phys_setup_not_lost_in_vlan_to_flat
  functional tests randomly fails

Status in neutron:
  New

Bug description:
  Seen 4 failures in last 8 days, 3 in master, 1 in stable/2023.2
  Fails as:-
  ft1.18: neutron.tests.functional.agent.test_l2_ovs_agent.TestOVSAgent.test_assert_pings_during_br_phys_setup_not_lost_in_vlan_to_flattesttools.testresult.real._StringException: Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 178, in func
      return f(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/test_l2_ovs_agent.py", line 270, in test_assert_pings_during_br_phys_setup_not_lost_in_vlan_to_flat
      self._test_assert_pings_during_br_phys_setup_not_lost(provider_net)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/test_l2_ovs_agent.py", line 305, in _test_assert_pings_during_br_phys_setup_not_lost
      with net_helpers.async_ping(ns_phys, [ip_int]) as done:
    File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
      next(self.gen)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/net_helpers.py", line 150, in async_ping
      f.result()
    File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
      return self.__get_result()
             ^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
      raise self._exception
    File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
      result = self.fn(*self.args, **self.kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/net_helpers.py", line 131, in assert_async_ping
      ns_ip_wrapper.netns.execute(
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/ip_lib.py", line 776, in execute
      return utils.execute(cmd, check_exit_code=check_exit_code,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py", line 157, in execute
      raise exceptions.ProcessExecutionError(msg,
  neutron_lib.exceptions.ProcessExecutionError: Exit code: 1; Cmd: ['ip', 'netns', 'exec', 'test-0245f548-3f29-48f0-8d28-0cfe7e68dcc9', 'ping', '-W', 1, '-c', '1', '10.148.117.171']; Stdin: ; Stdout: PING 10.148.117.171 (10.148.117.171) 56(84) bytes of data.

  --- 10.148.117.171 ping statistics ---
  1 packets transmitted, 0 received, 100% packet loss, time 0ms

  ; Stderr:


  Example failures:-
  https://03c08251732a708a286f-1284cae0458f2bcc716f47c02bf394ec.ssl.cf2.rackcdn.com/openstack/ffb419b2456b44899920be36e9599cc5/testr_results.html
  https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a5c/openstack/a5ca7963eaf34ac3846c631e3b52c272/testr_results.html
  https://d20c26a0475556dff012-de0170061a4c33caf35d62b404ee94ad.ssl.cf1.rackcdn.com/openstack/d17a470830d5471f8d6f309742a645fe/testr_results.html
  https://b19b5ab35f9ced465dd5-2eb50734132c0e56282483bcdf57bf8a.ssl.cf5.rackcdn.com/openstack/8a6caec32ede46adbdcc6aa3b9e8e1c1/testr_results.html

  Opensearch(creds: openstack/openstack):-
  https://opensearch.logs.openstack.org/_dashboards/app/data-explorer/discover?security_tenant=global#?_a=(discover:(columns:!(_source),isDirty:!f,sort:!()),metadata:(indexPattern:'94869730-aea8-11ec-9e6a-83741af3fdcd',view:discover))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-30d,to:now))&_q=(filters:!(),query:(language:kuery,query:'message:%22in%20test_assert_pings_during_br_phys_setup_not_lost_in_vlan_to_flat%22'))

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2107775/+subscriptions