yahoo-eng-team team mailing list archive

[Slawek Kaplonski <2007152@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

It seems that when there are 2 subnets using different segments created
on same network, it can cause race condition and one of the green
threads will expect tap port to be there but other will treat it as
"stale" device and will remove it from namespace.

It causes failures in the
neutron.tests.fullstack.test_multisegs.TestMultiSegs.test_multi_segs_network
pretty often.

Stacktrace example:

ft1.1: neutron.tests.fullstack.test_multisegs.TestMultiSegs.test_multi_segs_network(Open vSwitch Agent)testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
    eventlet.sleep(sleep)
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 36, in sleep
    hub.switch()
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch
    return self.greenlet.switch()
eventlet.timeout.Timeout: 60 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
    return f(self, *args, **kwargs)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_multisegs.py", line 146, in test_multi_segs_network
    self.vm1 = self._spawn_vm(neutron_port=self.port1)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_multisegs.py", line 65, in _spawn_vm
    vm.block_until_dhcp_config_done()
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/machine.py", line 217, in block_until_dhcp_config_done
    utils.wait_until_true(
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 747, in wait_until_true
    raise exception
neutron.tests.common.machine_fixtures.FakeMachineException: Address 10.0.11.195/24 or gateway 10.0.11.1 not configured properly on port portbc2128

Failure example:
https://750440148c983d142b4c-dd0601cbd7f0aa15c09a82437ec7e47b.ssl.cf5.rackcdn.com/872396/2/gate/neutron-
fullstack-with-uwsgi/c5e45ed/testr_results.html

** Affects: neutron
     Importance: Critical
     Assignee: Slawek Kaplonski (slaweq)
         Status: Confirmed


** Tags: gate-failure l3-ipam-dhcp

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2007152

Title:
  Race condition in DHCP agent cause deletion of the dhcp port as stale
  port

Status in neutron:
  Confirmed

Bug description:
  It seems that when there are 2 subnets using different segments
  created on same network, it can cause race condition and one of the
  green threads will expect tap port to be there but other will treat it
  as "stale" device and will remove it from namespace.

  It causes failures in the
  neutron.tests.fullstack.test_multisegs.TestMultiSegs.test_multi_segs_network
  pretty often.

  Stacktrace example:

  ft1.1: neutron.tests.fullstack.test_multisegs.TestMultiSegs.test_multi_segs_network(Open vSwitch Agent)testtools.testresult.real._StringException: Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true
      eventlet.sleep(sleep)
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 36, in sleep
      hub.switch()
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch
      return self.greenlet.switch()
  eventlet.timeout.Timeout: 60 seconds

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
      return f(self, *args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_multisegs.py", line 146, in test_multi_segs_network
      self.vm1 = self._spawn_vm(neutron_port=self.port1)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_multisegs.py", line 65, in _spawn_vm
      vm.block_until_dhcp_config_done()
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/machine.py", line 217, in block_until_dhcp_config_done
      utils.wait_until_true(
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 747, in wait_until_true
      raise exception
  neutron.tests.common.machine_fixtures.FakeMachineException: Address 10.0.11.195/24 or gateway 10.0.11.1 not configured properly on port portbc2128

  Failure example:
  https://750440148c983d142b4c-dd0601cbd7f0aa15c09a82437ec7e47b.ssl.cf5.rackcdn.com/872396/2/gate/neutron-
  fullstack-with-uwsgi/c5e45ed/testr_results.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2007152/+subscriptions