yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1257032] Re: nova makes calls to neutron with out considering URI size limit

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Venkata Siva Vijayendra Bhamidipati <vbhamidipati@xxxxxxxxxx>
Date: Thu, 12 Dec 2013 21:06:31 -0000
Reply-to: Bug 1257032 <1257032@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
*** This bug is a duplicate of bug 1228384 ***
    https://bugs.launchpad.net/bugs/1228384

Will close this bug as a dup of
https://bugs.launchpad.net/nova/+bug/1228384 . As part of that fix, Phil
has implemented chunking of server ids when querying neutron for ports
of VMs. The above stack trace is for a quantum deployment, and Phil's
patch will be ported over to that deployment.

** This bug has been marked a duplicate of bug 1228384
   Security Group extension reads all Neutron ports for anything other that a single server

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1257032

Title:
  nova makes calls to neutron with out considering URI size limit

Status in OpenStack Compute (Nova):
  New

Bug description:
  Nova when requesting security group instance bindings for servers
  makes a call to Neutron. When there are many servers, the URI size
  grows beyond 8k with neutron throwing up 414 error message.

  
  We could easily hit this bug in our deployment as we have several VMs running.

  There is a similar bug while making net-list. It needed to make subnet
  list internally and that would result in 414 if there are several
  subnets that can make URI size too long. But subnet-list was internal
  call and fix was appropriate in the neutron client.

  Here the bug is while fetching sg instance bindings which is primary
  call. Hence I feel the bug must be fixed in neutron API consumer which
  is nova.

  Also, there must be a general framework for all APIs to not to exceed
  URI size limit or fix all calls  with URIs which can extend beyond 8k
  size limit.

  Stacktrace for reference
  2013-11-27 13:06:01.696 ERROR nova.api.openstack [req-020f17cb-ee43-4cd2-a270-767936e6546b 6abe780581924062bdb648375abcb378 b9bbb06d41a942248e8d7070e17ed89d] Caught error: 414-{'message': ''}
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack Traceback (most recent call last):
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/__init__.py", line 81, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     return req.get_response(self.application)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/request.py", line 1053, in get_response
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     application, catch_exc_info=False)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/request.py", line 1022, in call_application
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 159, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     return resp(environ, start_response)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 450, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     return self.app(env, start_response)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 159, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     return resp(environ, start_response)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 159, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     return resp(environ, start_response)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 159, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     return resp(environ, start_response)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/Routes-1.12.3-py2.6.egg/routes/middleware.py", line 131, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     response = self.app(environ, start_response)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 159, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     return resp(environ, start_response)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 147, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 208, in call_func
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 890, in __call__
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     content_type, body, accept)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 969, in _process_stack
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     request, action_args)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 863, in post_process_extensions
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     **action_args)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/compute/contrib/security_groups.py", line 547, in detail
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     self._extend_servers(req, list(resp_obj.obj['servers']))
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/compute/contrib/security_groups.py", line 497, in _extend_servers
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     servers))
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/network/security_group/quantum_driver.py", line 258, in get_instances_security_groups_bindings
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     ports = quantum.list_ports(**search_opts).get('ports')
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 107, in with_params
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     ret = self.function(instance, *args, **kwargs)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 267, in list_ports
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     **_params)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 1018, in list
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     for r in self._pagination(collection, path, **params):
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 1031, in _pagination
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     res = self.get(path, params=params)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 1004, in get
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     headers=headers, params=params)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 989, in retry_request
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     headers=headers, params=params)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 934, in do_request
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     self._handle_fault_response(status_code, replybody)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 915, in _handle_fault_response
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     exception_handler_v20(status_code, des_error_body)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 92, in exception_handler_v20
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack     message=msg)
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack QuantumClientException: 414-{'message': ''}
  2013-11-27 13:06:01.696 30107 TRACE nova.api.openstack

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1257032/+subscriptions