yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1904897] [NEW] Failure in test_l2_agent_restart(OVS, Flat network)

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Terry Wilson <1904897@xxxxxxxxxxxxxxxxxx>
Date: Thu, 19 Nov 2020 16:49:40 -0000
Reply-to: Bug 1904897 <1904897@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Occasionally the test fails to ping. In this case, the tests failed on
an OVN-specific patch that could not have possibly interacted with the
test:
https://zuul.opendev.org/t/openstack/build/2eb4d891d3324814b28b1db5ea8ee148

{0}
neutron.tests.fullstack.test_connectivity.TestUninterruptedConnectivityOnL2AgentRestart.test_l2_agent_restart(OVS,Flat
network) [84.192269s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
    return f(self, *args, **kwargs)

      File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_connectivity.py", line 236, in test_l2_agent_restart
    self._assert_ping_during_agents_restart(

      File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/base.py", line 123, in _assert_ping_during_agents_restart
    common_utils.wait_until_true(

      File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)

      File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/net_helpers.py", line 145, in async_ping
    f.result()

      File "/usr/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()

      File "/usr/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception

      File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)

      File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/net_helpers.py", line 127, in assert_async_ping
    ns_ip_wrapper.netns.execute([ping_command, '-W', timeout, '-c', '1',

      File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/ip_lib.py", line 721, in execute
    return utils.execute(cmd, check_exit_code=check_exit_code,

      File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py", line 149, in execute
    raise exceptions.ProcessExecutionError(msg,

    neutron_lib.exceptions.ProcessExecutionError: Exit code: 1; Cmd:
['ip', 'netns', 'exec', 'test-dd15072f-a4b0-4568-82e9-51105e34b6a1',
'ping', '-W', 2, '-c', '1', '20.0.0.112']; Stdin: ; Stdout: PING
20.0.0.112 (20.0.0.112) 56(84) bytes of data.

--- 20.0.0.112 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

; Stderr:

There seem to be lots of dhcp agent errors here:
https://zuul.opendev.org/t/openstack/build/2eb4d891d3324814b28b1db5ea8ee148


like:

Unable to access
/tmp/tmp8ryzi43r/tmp_u0y3amv/state_path_c5hn5z2/external/pids/b8da6190-ce0d-4937-8346-678c702ab812.pid.haproxy;
Error: [Errno 2] No such file or directory:
'/tmp/tmp8ryzi43r/tmp_u0y3amv/state_path_c5hn5z2/external/pids/b8da6190-ce0d-4937-8346-678c702ab812.pid.haproxy'
get_value_from_file
/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py:251

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1904897

Title:
  Failure in test_l2_agent_restart(OVS,Flat network)

Status in neutron:
  New

Bug description:
  Occasionally the test fails to ping. In this case, the tests failed on
  an OVN-specific patch that could not have possibly interacted with the
  test:
  https://zuul.opendev.org/t/openstack/build/2eb4d891d3324814b28b1db5ea8ee148

  {0}
  neutron.tests.fullstack.test_connectivity.TestUninterruptedConnectivityOnL2AgentRestart.test_l2_agent_restart(OVS,Flat
  network) [84.192269s] ... FAILED

  Captured traceback:
  ~~~~~~~~~~~~~~~~~~~
      Traceback (most recent call last):

        File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
      return f(self, *args, **kwargs)

        File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_connectivity.py", line 236, in test_l2_agent_restart
      self._assert_ping_during_agents_restart(

        File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/base.py", line 123, in _assert_ping_during_agents_restart
      common_utils.wait_until_true(

        File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
      next(self.gen)

        File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/net_helpers.py", line 145, in async_ping
      f.result()

        File "/usr/lib/python3.8/concurrent/futures/_base.py", line 432, in result
      return self.__get_result()

        File "/usr/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
      raise self._exception

        File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
      result = self.fn(*self.args, **self.kwargs)

        File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/net_helpers.py", line 127, in assert_async_ping
      ns_ip_wrapper.netns.execute([ping_command, '-W', timeout, '-c', '1',

        File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/ip_lib.py", line 721, in execute
      return utils.execute(cmd, check_exit_code=check_exit_code,

        File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py", line 149, in execute
      raise exceptions.ProcessExecutionError(msg,

      neutron_lib.exceptions.ProcessExecutionError: Exit code: 1; Cmd:
  ['ip', 'netns', 'exec', 'test-dd15072f-a4b0-4568-82e9-51105e34b6a1',
  'ping', '-W', 2, '-c', '1', '20.0.0.112']; Stdin: ; Stdout: PING
  20.0.0.112 (20.0.0.112) 56(84) bytes of data.

  --- 20.0.0.112 ping statistics ---
  1 packets transmitted, 0 received, 100% packet loss, time 0ms

  ; Stderr:

  There seem to be lots of dhcp agent errors here:
  https://zuul.opendev.org/t/openstack/build/2eb4d891d3324814b28b1db5ea8ee148

  
  like:

  Unable to access
  /tmp/tmp8ryzi43r/tmp_u0y3amv/state_path_c5hn5z2/external/pids/b8da6190-ce0d-4937-8346-678c702ab812.pid.haproxy;
  Error: [Errno 2] No such file or directory:
  '/tmp/tmp8ryzi43r/tmp_u0y3amv/state_path_c5hn5z2/external/pids/b8da6190-ce0d-4937-8346-678c702ab812.pid.haproxy'
  get_value_from_file
  /home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py:251

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1904897/+subscriptions
Follow ups

[Bug 1904897] Re: Failure in test_l2_agent_restart(OVS, Flat network)
From: Rodolfo Alonso, 2022-11-30