yahoo-eng-team team mailing list archive

[Slawek Kaplonski <1915271@xxxxxxxxxxxxxxxxxx>]

** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1915271

Title:
  test_create_router_set_gateway_with_fixed_ip fails in dvr-ha job

Status in neutron:
  Fix Released

Bug description:
  test_create_router_set_gateway_with_fixed_ip periodically fails in neutron-tempest-dvr-ha-multinode-full.
  Failure start to happen after new engine facade switch in L3 code.
  Failure is due to router failed to be deleted (neutron returns 500), see first line in below traceback.

  Traceback:

  2021-01-28 13:36:26,139 81948 INFO     [tempest.lib.common.rest_client] Request (RoutersAdminTest:test_create_router_set_gateway_with_fixed_ip): 500 DELETE https://10.209.160.170:9696/v2.0/routers/3f337e0e-3bed-44f7-a8f9-e43c01787445 10.779s
  2021-01-28 13:36:26,139 81948 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
          Body: None
      Response - Headers: {'date': 'Thu, 28 Jan 2021 13:36:26 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-type': 'application/json', 'content-length': '150', 'x-openstack-request-id': 'req-2392442f-a3b0-4c79-a79c-c765c7cab834', 'connection': 'close', 'status': '500', 'content-location': 'https://10.209.160.170:9696/v2.0/routers/3f337e0e-3bed-44f7-a8f9-e43c01787445'}
          Body: b'{"NeutronError": {"type": "HTTPInternalServerError", "message": "Request Failed: internal server error while processing your request.", "detail": ""}}'
  2021-01-28 13:36:26,305 81948 INFO     [tempest.lib.common.rest_client] Request (RoutersAdminTest:_run_cleanups): 409 DELETE https://10.209.160.170:9696/v2.0/subnets/812a9855-15a2-4a8e-b246-c6ea68cdadcd 0.165s
  2021-01-28 13:36:26,306 81948 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
          Body: None
      Response - Headers: {'date': 'Thu, 28 Jan 2021 13:36:26 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-type': 'application/json', 'content-length': '204', 'x-openstack-request-id': 'req-9eb0af8b-9154-489b-94b8-93d3da458c05', 'connection': 'close', 'status': '409', 'content-location': 'https://10.209.160.170:9696/v2.0/subnets/812a9855-15a2-4a8e-b246-c6ea68cdadcd'}
          Body: b'{"NeutronError": {"type": "SubnetInUse", "message": "Unable to complete operation on subnet 812a9855-15a2-4a8e-b246-c6ea68cdadcd: One or more ports have an IP allocation from this subnet.", "detail": ""}}'
  2021-01-28 13:36:26,544 81948 INFO     [tempest.lib.common.rest_client] Request (RoutersAdminTest:_run_cleanups): 409 DELETE https://10.209.160.170:9696/v2.0/networks/c06258a3-d817-4ffd-b9c6-1c20eaedd688 0.238s
  2021-01-28 13:36:26,545 81948 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
          Body: None
      Response - Headers: {'date': 'Thu, 28 Jan 2021 13:36:26 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-type': 'application/json', 'content-length': '205', 'x-openstack-request-id': 'req-a412a0f0-832c-45f4-9e9d-40deef43cde2', 'connection': 'close', 'status': '409', 'content-location': 'https://10.209.160.170:9696/v2.0/networks/c06258a3-d817-4ffd-b9c6-1c20eaedd688'}
          Body: b'{"NeutronError": {"type": "NetworkInUse", "message": "Unable to complete operation on network c06258a3-d817-4ffd-b9c6-1c20eaedd688. There are one or more ports still in use on the network.", "detail": ""}}'
  }}}

  traceback-1: {{{
  Traceback (most recent call last):
    File "/opt/stack/tempest/tempest/lib/common/utils/test_utils.py", line 87, in call_and_ignore_notfound_exc
      return func(*args, **kwargs)
    File "/opt/stack/tempest/tempest/lib/services/network/subnets_client.py", line 52, in delete_subnet
      return self.delete_resource(uri)
    File "/opt/stack/tempest/tempest/lib/services/network/base.py", line 41, in delete_resource
      resp, body = self.delete(req_uri)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 331, in delete
      return self.request('DELETE', url, extra_headers, headers, body)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 704, in request
      self._error_checker(resp, resp_body)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 825, in _error_checker
      raise exceptions.Conflict(resp_body, resp=resp)
  tempest.lib.exceptions.Conflict: Conflict with state of target resource
  Details: {'type': 'SubnetInUse', 'message': 'Unable to complete operation on subnet 812a9855-15a2-4a8e-b246-c6ea68cdadcd: One or more ports have an IP allocation from this subnet.', 'detail': ''}
  }}}

  traceback-2: {{{
  Traceback (most recent call last):
    File "/opt/stack/tempest/tempest/lib/common/utils/test_utils.py", line 87, in call_and_ignore_notfound_exc
      return func(*args, **kwargs)
    File "/opt/stack/tempest/tempest/lib/services/network/networks_client.py", line 52, in delete_network
      return self.delete_resource(uri)
    File "/opt/stack/tempest/tempest/lib/services/network/base.py", line 41, in delete_resource
      resp, body = self.delete(req_uri)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 331, in delete
      return self.request('DELETE', url, extra_headers, headers, body)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 704, in request
      self._error_checker(resp, resp_body)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 825, in _error_checker
      raise exceptions.Conflict(resp_body, resp=resp)
  tempest.lib.exceptions.Conflict: Conflict with state of target resource
  Details: {'type': 'NetworkInUse', 'message': 'Unable to complete operation on network c06258a3-d817-4ffd-b9c6-1c20eaedd688. There are one or more ports still in use on the network.', 'detail': ''}
  }}}

  Traceback (most recent call last):
    File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 109, in wrapper
      return func(*func_args, **func_kwargs)
    File "/opt/stack/tempest/tempest/api/network/admin/test_routers.py", line 254, in test_create_router_set_gateway_with_fixed_ip
      self.admin_routers_client.delete_router(router['id'])
    File "/opt/stack/tempest/tempest/lib/services/network/routers_client.py", line 52, in delete_router
      return self.delete_resource(uri)
    File "/opt/stack/tempest/tempest/lib/services/network/base.py", line 41, in delete_resource
      resp, body = self.delete(req_uri)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 331, in delete
      return self.request('DELETE', url, extra_headers, headers, body)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 704, in request
      self._error_checker(resp, resp_body)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 880, in _error_checker
      raise exceptions.ServerFault(resp_body, resp=resp,
  tempest.lib.exceptions.ServerFault: Got server fault
  Details: Request Failed: internal server error while processing your request.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1915271/+subscriptions