yahoo-eng-team team mailing list archive

[Ihar Hrachyshka <1707933@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Grafana failure rate at 30%; triage of July gate failures for the job
suggested that most of them come from the same root cause: some test
workers killed during test execution:

https://etherpad.openstack.org/p/neutron-functional-gate-failures-july

When it happens, we see the following line in console log:

2017-07-13 17:54:12.742494 | 2017-07-13 17:54:12.742 | Killed

Most of the times it happens during FirewallTestCase execution, which is
suspicious.

Kuba sent a test patch to collect information about callers to 'kill'
CLI: https://review.openstack.org/#/c/487065/ but it hasn't revealed
much.

** Affects: neutron
     Importance: Critical
         Status: Confirmed


** Tags: functional-tests gate-failure

** Changed in: neutron
   Importance: Undecided => Critical

** Changed in: neutron
       Status: New => Confirmed

** Tags added: gate

** Tags removed: gate
** Tags added: functional-tests gate-failure

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1707933

Title:
  functional tests timeout after a test worker killed

Status in neutron:
  Confirmed

Bug description:
  Grafana failure rate at 30%; triage of July gate failures for the job
  suggested that most of them come from the same root cause: some test
  workers killed during test execution:

  https://etherpad.openstack.org/p/neutron-functional-gate-failures-july

  When it happens, we see the following line in console log:

  2017-07-13 17:54:12.742494 | 2017-07-13 17:54:12.742 | Killed

  Most of the times it happens during FirewallTestCase execution, which
  is suspicious.

  Kuba sent a test patch to collect information about callers to 'kill'
  CLI: https://review.openstack.org/#/c/487065/ but it hasn't revealed
  much.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1707933/+subscriptions