yahoo-eng-team team mailing list archive

[Slawek Kaplonski <1843285@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

This scenario test neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_subport_connectivity is failing quite often in neutron-tempest-plugin-dvr-multinode-scenario-queens job.
But it's not failing in same job run on rocky, stein or master branch. The difference between those jobs is that in queens we are still using iptables_hybrid fw driver and in newer versions we are using ovs based firewall.

Logstash query:
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20279%2C%20in%20test_subport_connectivity%5C%22

Example of failure:

2019-09-08 23:09:16.496631 | controller | {0} neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_subport_connectivity [446.949502s] ... FAILED
2019-09-08 23:09:16.496896 | controller |
2019-09-08 23:09:16.496944 | controller | Captured traceback:
2019-09-08 23:09:16.496985 | controller | ~~~~~~~~~~~~~~~~~~~
2019-09-08 23:09:16.497041 | controller |     Traceback (most recent call last):
2019-09-08 23:09:16.497177 | controller |       File "/opt/stack/tempest/.tox/tempest/local/lib/python2.7/site-packages/neutron_tempest_plugin/scenario/test_trunk.py", line 279, in test_subport_connectivity
2019-09-08 23:09:16.497298 | controller |         should_succeed=False)
2019-09-08 23:09:16.497424 | controller |       File "/opt/stack/tempest/.tox/tempest/local/lib/python2.7/site-packages/neutron_tempest_plugin/scenario/base.py", line 308, in check_remote_connectivity
2019-09-08 23:09:16.497468 | controller |         timeout=timeout))
2019-09-08 23:09:16.498033 | controller |       File "/opt/stack/tempest/.tox/tempest/local/lib/python2.7/site-packages/unittest2/case.py", line 702, in assertTrue
2019-09-08 23:09:16.498089 | controller |         raise self.failureException(msg)
2019-09-08 23:09:16.498153 | controller |     AssertionError: False is not true


We have same issues in our d/s CI also on versions newer than queens but we are using there iptables_hybrid fw driver also.
So it looks for me that there might be some race condition in iptables_hybrid fw driver and trunk ports configuration.

** Affects: neutron
     Importance: High
         Status: Confirmed


** Tags: gate-failure trunk

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1843285

Title:
  Trunk scenario test test_subport_connectivity failing with
  iptables_hybrid fw driver

Status in neutron:
  Confirmed

Bug description:
  This scenario test neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_subport_connectivity is failing quite often in neutron-tempest-plugin-dvr-multinode-scenario-queens job.
  But it's not failing in same job run on rocky, stein or master branch. The difference between those jobs is that in queens we are still using iptables_hybrid fw driver and in newer versions we are using ovs based firewall.

  Logstash query:
  http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20279%2C%20in%20test_subport_connectivity%5C%22

  Example of failure:

  2019-09-08 23:09:16.496631 | controller | {0} neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_subport_connectivity [446.949502s] ... FAILED
  2019-09-08 23:09:16.496896 | controller |
  2019-09-08 23:09:16.496944 | controller | Captured traceback:
  2019-09-08 23:09:16.496985 | controller | ~~~~~~~~~~~~~~~~~~~
  2019-09-08 23:09:16.497041 | controller |     Traceback (most recent call last):
  2019-09-08 23:09:16.497177 | controller |       File "/opt/stack/tempest/.tox/tempest/local/lib/python2.7/site-packages/neutron_tempest_plugin/scenario/test_trunk.py", line 279, in test_subport_connectivity
  2019-09-08 23:09:16.497298 | controller |         should_succeed=False)
  2019-09-08 23:09:16.497424 | controller |       File "/opt/stack/tempest/.tox/tempest/local/lib/python2.7/site-packages/neutron_tempest_plugin/scenario/base.py", line 308, in check_remote_connectivity
  2019-09-08 23:09:16.497468 | controller |         timeout=timeout))
  2019-09-08 23:09:16.498033 | controller |       File "/opt/stack/tempest/.tox/tempest/local/lib/python2.7/site-packages/unittest2/case.py", line 702, in assertTrue
  2019-09-08 23:09:16.498089 | controller |         raise self.failureException(msg)
  2019-09-08 23:09:16.498153 | controller |     AssertionError: False is not true

  
  We have same issues in our d/s CI also on versions newer than queens but we are using there iptables_hybrid fw driver also.
  So it looks for me that there might be some race condition in iptables_hybrid fw driver and trunk ports configuration.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1843285/+subscriptions