yahoo-eng-team team mailing list archive

[Slawek Kaplonski <1875344@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

I saw it couple of times in our d/s testing job with DVR enabled that
test
neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_delete_policies_while_tenant_attached_to_net
was failing in cleanup phase.

Error was like 
ft4.1: tearDownClass (neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON)testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tempest/test.py", line 242, in tearDownClass
    six.reraise(etype, value, trace)
  File "/usr/lib/python3.6/site-packages/six.py", line 675, in reraise
    raise value
  File "/usr/lib/python3.6/site-packages/tempest/test.py", line 214, in tearDownClass
    teardown()
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/api/base.py", line 196, in resource_cleanup
    subnet['id'])
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/api/base.py", line 281, in _try_delete_resource
    delete_callable(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/services/network/json/network_client.py", line 112, in _delete
    resp, body = self.delete(uri)
  File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 314, in delete
    return self.request('DELETE', url, extra_headers, headers, body)
  File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 687, in request
    self._error_checker(resp, resp_body)
  File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 808, in _error_checker
    raise exceptions.Conflict(resp_body, resp=resp)
tempest.lib.exceptions.Conflict: Conflict with state of target resource
Details: {'type': 'SubnetInUse', 'message': 'Unable to complete operation on subnet 7f774581-bb05-4c71-ac4e-1a338c1e3fa1: One or more ports have an IP allocation from this subnet.', 'detail': ''}

The issue is that in the dvr deployment, when router with external
network is created, Neutron creates in the background floatingip gateway
port. And as test is performing fast it may happen that this port is
created after router is already deleted so it's not cleaned properly and
causes failure during network removal.

We didn't saw it in u/s CI as we don't have any dvr based job which
would run neutron_tempest_plugin.api tests.

** Affects: neutron
     Importance: Medium
     Assignee: Slawek Kaplonski (slaweq)
         Status: Confirmed


** Tags: tempest

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1875344

Title:
  Cleanup in
  neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON
  may fail in dvr deployments

Status in neutron:
  Confirmed

Bug description:
  I saw it couple of times in our d/s testing job with DVR enabled that
  test
  neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_delete_policies_while_tenant_attached_to_net
  was failing in cleanup phase.

  Error was like 
  ft4.1: tearDownClass (neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON)testtools.testresult.real._StringException: Traceback (most recent call last):
    File "/usr/lib/python3.6/site-packages/tempest/test.py", line 242, in tearDownClass
      six.reraise(etype, value, trace)
    File "/usr/lib/python3.6/site-packages/six.py", line 675, in reraise
      raise value
    File "/usr/lib/python3.6/site-packages/tempest/test.py", line 214, in tearDownClass
      teardown()
    File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/api/base.py", line 196, in resource_cleanup
      subnet['id'])
    File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/api/base.py", line 281, in _try_delete_resource
      delete_callable(*args, **kwargs)
    File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/services/network/json/network_client.py", line 112, in _delete
      resp, body = self.delete(uri)
    File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 314, in delete
      return self.request('DELETE', url, extra_headers, headers, body)
    File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 687, in request
      self._error_checker(resp, resp_body)
    File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 808, in _error_checker
      raise exceptions.Conflict(resp_body, resp=resp)
  tempest.lib.exceptions.Conflict: Conflict with state of target resource
  Details: {'type': 'SubnetInUse', 'message': 'Unable to complete operation on subnet 7f774581-bb05-4c71-ac4e-1a338c1e3fa1: One or more ports have an IP allocation from this subnet.', 'detail': ''}

  The issue is that in the dvr deployment, when router with external
  network is created, Neutron creates in the background floatingip
  gateway port. And as test is performing fast it may happen that this
  port is created after router is already deleted so it's not cleaned
  properly and causes failure during network removal.

  We didn't saw it in u/s CI as we don't have any dvr based job which
  would run neutron_tempest_plugin.api tests.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1875344/+subscriptions