← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #53121

[Bug 1586931] Re: TestServerBasicOps: Test fails when deleting server and floating ip almost at the same time

  Thread Previous • Date Previous • • Thread Next 
I hit this. I can reproduce it almost every time on my env using
linuxbrige+vxlan. The nova trace is:

2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions [req-bf41dac1-8fc0-4fd6-9a35-d754cea79057 9a0be6e4b8bf4cadb4a43401696fec19 48935f9a5ed84703973c70dd70859b7f - - -] Unexpected exception in API method
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/floating_ips.py", line 173, in delete
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     context, instance, floating_ip)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1527, in disassociate_and_release_floating_ip
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     raise_if_associated=False)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1536, in _release_floating_ip
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     client.delete_floatingip(fip['id'])
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 102, in with_params
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     ret = self.function(instance, *args, **kwargs)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 751, in delete_floatingip
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     return self.delete(self.floatingip_path % (floatingip))
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 289, in delete
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     headers=headers, params=params)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 270, in retry_request
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     headers=headers, params=params)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 211, in do_request
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     self._handle_fault_response(status_code, replybody)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 185, in _handle_fault_response
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     exception_handler_v20(status_code, des_error_body)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 70, in exception_handler_v20
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions     status_code=status_code)
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions PortNotFoundClient: Port f4f11381-dc3b-41b2-94ca-4a9f494c0372 could not be found.
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions

2 operation occurs: the VM deletion (that triggers the Neutron port
deletion) and the floating ip deletion. Nova sends a request to Neutron
to delete the floating IP. When a floating IP is deleted Neutron will
get the port associated with the floating ip to send a  network change
event notification to Nova. The get port fails with PortNotFound because
in the meanwhile the Neutron port that the VM was using, has been
deleted. The floating ip request fails because Neutron sends back to
Nova a PortNotFound error.

** Also affects: neutron
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1586931

Title:
  TestServerBasicOps: Test fails when deleting server and floating ip
  almost at the same time

Status in neutron:
  New
Status in OpenStack Compute (nova):
  Incomplete
Status in tempest:
  In Progress

Bug description:
  In
  tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops,
  after last step:
  self.servers_client.delete_server(self.instance['id']) it doesn't wait
  for the server to be deleted, and then deletes the floating ip
  immediately in the clean up, this will cause faiure:

  Here is the partial log:
  2016-05-29 21:51:29.499 29791 INFO tempest.lib.common.rest_client [req-c3588ac4-21ca-47c3-bdb1-62088efd7a8b ] Request (TestServerBasicOps:test_server_basic_ops): 204 DELETE https://<url>:8774/v2/159886ce087a4f8fbfbcab14947d96b1/servers/6d44763b-ea79-4b5b-b57e-714191802c7c 0.465s
  2016-05-29 21:51:29.499 29791 DEBUG tempest.lib.common.rest_client [req-c3588ac4-21ca-47c3-bdb1-62088efd7a8b ] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
          Body: None
      Response - Headers: {'status': '204', 'content-length': '0', 'content-location': 'https://<url>:8774/v2/159886ce087a4f8fbfbcab14947d96b1/servers/6d44763b-ea79-4b5b-b57e-714191802c7c', 'date': 'Mon, 30 May 2016 02:51:29 GMT', 'x-compute-request-id': 'req-c3588ac4-21ca-47c3-bdb1-62088efd7a8b', 'content-type': 'application/json', 'connection': 'close'}
          Body:  _log_request_full tempest/lib/common/rest_client.py:422
  2016-05-29 21:51:30.410 29791 INFO tempest.lib.common.rest_client [req-db2323f5-3d58-4fd7-ae51-44f5525c6689 ] Request (TestServerBasicOps:_run_cleanups): 500 DELETE https://<url>:8774/v2/159886ce087a4f8fbfbcab14947d96b1/os-floating-ips/948912f6-ce03-4856-922b-59c4f16d3740 0.910s
  2016-05-29 21:51:30.410 29791 DEBUG tempest.lib.common.rest_client [req-db2323f5-3d58-4fd7-ae51-44f5525c6689 ] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
          Body: None
      Response - Headers: {'status': '500', 'content-length': '224', 'content-location': 'https://<url>:8774/v2/159886ce087a4f8fbfbcab14947d96b1/os-floating-ips/948912f6-ce03-4856-922b-59c4f16d3740', 'date': 'Mon, 30 May 2016 02:51:30 GMT', 'x-compute-request-id': 'req-db2323f5-3d58-4fd7-ae51-44f5525c6689', 'content-type': 'application/json; charset=UTF-8', 'connection': 'close'}
          Body: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'neutronclient.common.exceptions.PortNotFoundClient'>", "code": 500}}

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1586931/+subscriptions
  Thread Previous • Date Previous • • Thread Next