yahoo-eng-team team mailing list archive

[Bernard Cafarelli <bcafarel@xxxxxxxxxx>]

Public bug reported:

On my CentOS system, this test can locally fail 30-50% of the time - tested from queens to master:
==============================
Failed 1 tests - output below:
==============================

neutron.tests.unit.scheduler.test_dhcp_agent_scheduler.TestNetworksFailover.test_filter_bindings
------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "neutron/tests/base.py", line 177, in func
        return f(self, *args, **kwargs)
      File "neutron/tests/unit/scheduler/test_dhcp_agent_scheduler.py", line 527, in test_filter_bindings
        self.assertIn(network_ids[2], res_ids)
      File "/home/stack/neutron/.tox/py27/lib/python2.7/site-packages/testtools/testcase.py", line 417, in assertIn
        self.assertThat(haystack, Contains(needle), message)
      File "/home/stack/neutron/.tox/py27/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
        raise mismatch_error
    testtools.matchers._impl.MismatchError: '8a6b1ce0-fc2e-40b0-bc98-a59722d96a3f' not in ['6f6c1774-b351-4763-a8d8-184c3bdf01de', '8cee4a38-63e8-4c71-99e9-611b61aa4c90']


The test creates 4 networks (random UUIDs), then 4 NetworkDhcpAgentBindings,
It then gets a list of these bindings with NetworkDhcpAgentBinding.get_objects() which uses a sort based on network_id by default.
This list is then used with _filter_bindings:
          with mock.patch.object(self, 'agent_starting_up',           
                                 side_effect=[True, False]):                     
              res = [b for b in self._filter_bindings(None, bindings_objs)]
[...]
              res_ids = [b.network_id for b in res]                     
              self.assertIn(network_ids[2], res_ids)                
              self.assertIn(network_ids[3], res_ids)

But as network_ids is not sorted, this can fail depending on the generated UUIDs. One example on my system:
network_ids:
['6f6c1774-b351-4763-a8d8-184c3bdf01de', '8cee4a38-63e8-4c71-99e9-611b61aa4c90', '8a6b1ce0-fc2e-40b0-bc98-a59722d96a3f', '4f476b11-085f-47bf-a27f-e300eb9a85b4']

binding_objs:
NetworkDhcpAgentBinding(dhcp_agent_id=1d6b3b74-afa1-410b-880e-be4f87ce7c6d,network_id=4f476b11-085f-47bf-a27f-e300eb9a85b4)
NetworkDhcpAgentBinding(dhcp_agent_id=07030f55-35cd-4386-853f-70777cdcae2b,network_id=6f6c1774-b351-4763-a8d8-184c3bdf01de)
NetworkDhcpAgentBinding(dhcp_agent_id=1d6b3b74-afa1-410b-880e-be4f87ce7c6d,network_id=8a6b1ce0-fc2e-40b0-bc98-a59722d96a3f)
NetworkDhcpAgentBinding(dhcp_agent_id=07030f55-35cd-4386-853f-70777cdcae2b,network_id=8cee4a38-63e8-4c71-99e9-611b61aa4c90)

which will give a (failing the test) res_ids:
['6f6c1774-b351-4763-a8d8-184c3bdf01de',
'8cee4a38-63e8-4c71-99e9-611b61aa4c90']

I am not sure why upstream gates never seem to have the problem (as far
as I have checked), but sorting the network_ids makes the test passing
all the time

** Affects: neutron
     Importance: Undecided
     Assignee: Bernard Cafarelli (bcafarel)
         Status: New


** Tags: unittest

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1839595

Title:
  neutron.tests.unit.scheduler.test_dhcp_agent_scheduler.TestNetworksFailover.test_filter_bindings
  test can fail depending on generated UUIDs

Status in neutron:
  New

Bug description:
  On my CentOS system, this test can locally fail 30-50% of the time - tested from queens to master:
  ==============================
  Failed 1 tests - output below:
  ==============================

  neutron.tests.unit.scheduler.test_dhcp_agent_scheduler.TestNetworksFailover.test_filter_bindings
  ------------------------------------------------------------------------------------------------

  Captured traceback:
  ~~~~~~~~~~~~~~~~~~~
      Traceback (most recent call last):
        File "neutron/tests/base.py", line 177, in func
          return f(self, *args, **kwargs)
        File "neutron/tests/unit/scheduler/test_dhcp_agent_scheduler.py", line 527, in test_filter_bindings
          self.assertIn(network_ids[2], res_ids)
        File "/home/stack/neutron/.tox/py27/lib/python2.7/site-packages/testtools/testcase.py", line 417, in assertIn
          self.assertThat(haystack, Contains(needle), message)
        File "/home/stack/neutron/.tox/py27/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
          raise mismatch_error
      testtools.matchers._impl.MismatchError: '8a6b1ce0-fc2e-40b0-bc98-a59722d96a3f' not in ['6f6c1774-b351-4763-a8d8-184c3bdf01de', '8cee4a38-63e8-4c71-99e9-611b61aa4c90']

  
  The test creates 4 networks (random UUIDs), then 4 NetworkDhcpAgentBindings,
  It then gets a list of these bindings with NetworkDhcpAgentBinding.get_objects() which uses a sort based on network_id by default.
  This list is then used with _filter_bindings:
            with mock.patch.object(self, 'agent_starting_up',           
                                   side_effect=[True, False]):                     
                res = [b for b in self._filter_bindings(None, bindings_objs)]
  [...]
                res_ids = [b.network_id for b in res]                     
                self.assertIn(network_ids[2], res_ids)                
                self.assertIn(network_ids[3], res_ids)

  But as network_ids is not sorted, this can fail depending on the generated UUIDs. One example on my system:
  network_ids:
  ['6f6c1774-b351-4763-a8d8-184c3bdf01de', '8cee4a38-63e8-4c71-99e9-611b61aa4c90', '8a6b1ce0-fc2e-40b0-bc98-a59722d96a3f', '4f476b11-085f-47bf-a27f-e300eb9a85b4']

  binding_objs:
  NetworkDhcpAgentBinding(dhcp_agent_id=1d6b3b74-afa1-410b-880e-be4f87ce7c6d,network_id=4f476b11-085f-47bf-a27f-e300eb9a85b4)
  NetworkDhcpAgentBinding(dhcp_agent_id=07030f55-35cd-4386-853f-70777cdcae2b,network_id=6f6c1774-b351-4763-a8d8-184c3bdf01de)
  NetworkDhcpAgentBinding(dhcp_agent_id=1d6b3b74-afa1-410b-880e-be4f87ce7c6d,network_id=8a6b1ce0-fc2e-40b0-bc98-a59722d96a3f)
  NetworkDhcpAgentBinding(dhcp_agent_id=07030f55-35cd-4386-853f-70777cdcae2b,network_id=8cee4a38-63e8-4c71-99e9-611b61aa4c90)

  which will give a (failing the test) res_ids:
  ['6f6c1774-b351-4763-a8d8-184c3bdf01de',
  '8cee4a38-63e8-4c71-99e9-611b61aa4c90']

  I am not sure why upstream gates never seem to have the problem (as
  far as I have checked), but sorting the network_ids makes the test
  passing all the time

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1839595/+subscriptions