yahoo-eng-team team mailing list archive

[OpenStack Infra <2070390@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/922786
Committed: https://opendev.org/openstack/neutron/commit/1c888c94a3457c5bc2225484e5dd86978c3d265d
Submitter: "Zuul (22348)"
Branch:    master

commit 1c888c94a3457c5bc2225484e5dd86978c3d265d
Author: Gaudenz Steinlin <gaudenz.steinlin@xxxxxxxxxxxxx>
Date:   Wed Jun 26 09:32:59 2024 +0200

    Improve Process fixture service restart handling
    
    The test_l2_agent_restart test was failing due to the agents not
    restarting within the timeout of 30s. This is fixed by:
    
    * Use `systemctl restart` to restart the service instead of killing
      and creating a  new transient service.
    * Don't block on `systemctl` calls to allow parallel service
      operations. Previously this was serialized in the rootwrap daemon
      which lead to delays.
    * Use `KillMode=mixed` to first only kill the main process and give it
      25s to cleanly shutdown all other processes. After this timeout all
      processes are killed. Previously systemd sent a SIGTERM to all
      processes which caused unclean shutdowns of some neutron agents which
      expected to shutdown their child processes themselves.
    
    Change-Id: Ic752e36e6fe6ba9b1fc9e7296204c086c465d76f
    Closes-Bug: #2070390


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2070390

Title:
  [fullstack][dalmatian] Random failures in test_l2_agent_restart

Status in neutron:
  Fix Released

Bug description:
  Fails like:-
  ft1.1: neutron.tests.fullstack.test_connectivity.TestUninterruptedConnectivityOnL2AgentRestartOvs.test_l2_agent_restart(OVS,Flat network)testtools.testresult.real._StringException: Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 178, in func
      return f(self, *args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_connectivity.py", line 179, in test_l2_agent_restart
      self._test_l2_agent_restart(agent_restart_timeout)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_connectivity.py", line 162, in _test_l2_agent_restart
      self._assert_ping_during_agents_restart(
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/base.py", line 135, in _assert_ping_during_agents_restart
      self.assertTrue(all(r.done() for r in restarts))
    File "/usr/lib/python3.10/unittest/case.py", line 687, in assertTrue
      raise self.failureException(msg)
  AssertionError: False is not true

  Sample failures:-
   - https://86ddc5b3525cc8a57bd5-84709aab2060acc1565f07c661aff448.ssl.cf1.rackcdn.com/922119/1/check/neutron-fullstack-with-uwsgi/4213115/testr_results.html

   -
  https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e5d/888098/15/check/neutron-
  fullstack-with-uwsgi/e5d8147/testr_results.html

   -
  https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6c6/922470/1/check/neutron-
  fullstack-with-uwsgi/6c63681/testr_results.html

   -
  https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_522/periodic/opendev.org/openstack/neutron/master/neutron-
  fullstack/52235b5/testr_results.html

   -
  https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_14b/922074/4/check/neutron-
  fullstack-with-uwsgi/14bece8/testr_results.html

  
  Triggered by https://review.opendev.org/c/openstack/neutron/+/919834

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2070390/+subscriptions