← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #30587

[Bug 1437863] [NEW] API: doing "nova list" when neutron service is down has an internal error

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

nicira@Ubuntu1204Server:~$ nova list
ERROR (ClientException): The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-014bcadf-317f-4d2f-b0a0-dbebfc90bedf)

The trace from the API is below

2015-03-29 06:50:05.244 ERROR nova.api.openstack [req-014bcadf-317f-4d2f-b0a0-dbebfc90bedf demo demo] Caught error: Unable to establish connection to http://10.0.0.40:9696/v2.0/ports.json?device_id=be9cd6ae-091e-47ba-b3f8-4139b6326df2
2015-03-29 06:50:05.244 TRACE nova.api.openstack Traceback (most recent call last):
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/__init__.py", line 125, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return req.get_response(self.application)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
2015-03-29 06:50:05.244 TRACE nova.api.openstack     application, catch_exc_info=False)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
2015-03-29 06:50:05.244 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return resp(environ, start_response)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 634, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self._call_app(env, start_response)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 554, in _call_app
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self._app(env, _fake_start_response)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return resp(environ, start_response)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return resp(environ, start_response)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 136, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     response = self.app(environ, start_response)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return resp(environ, start_response)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 756, in __call__
2015-03-29 06:50:05.244 TRACE nova.api.openstack     content_type, body, accept)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 847, in _process_stack
2015-03-29 06:50:05.244 TRACE nova.api.openstack     request, action_args)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 710, in post_process_extensions
2015-03-29 06:50:05.244 TRACE nova.api.openstack     **action_args)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/security_groups.py", line 471, in detail
2015-03-29 06:50:05.244 TRACE nova.api.openstack     self._extend_servers(req, list(resp_obj.obj['servers']))
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/security_groups.py", line 422, in _extend_servers
2015-03-29 06:50:05.244 TRACE nova.api.openstack     servers))
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/security_group/neutron_driver.py", line 353, in get_instances_security_groups_bindings
2015-03-29 06:50:05.244 TRACE nova.api.openstack     ports = self._get_ports_from_server_list(servers, neutron)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/security_group/neutron_driver.py", line 312, in _get_ports_from_server_list
2015-03-29 06:50:05.244 TRACE nova.api.openstack     ports.extend(neutron.list_ports(**search_opts).get('ports'))
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 99, in with_params
2015-03-29 06:50:05.244 TRACE nova.api.openstack     ret = self.function(instance, *args, **kwargs)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 497, in list_ports
2015-03-29 06:50:05.244 TRACE nova.api.openstack     **_params)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 304, in list
2015-03-29 06:50:05.244 TRACE nova.api.openstack     for r in self._pagination(collection, path, **params):
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 317, in _pagination
2015-03-29 06:50:05.244 TRACE nova.api.openstack     res = self.get(path, params=params)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 290, in get
2015-03-29 06:50:05.244 TRACE nova.api.openstack     headers=headers, params=params)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 267, in retry_request
2015-03-29 06:50:05.244 TRACE nova.api.openstack     headers=headers, params=params)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 197, in do_request
2015-03-29 06:50:05.244 TRACE nova.api.openstack     content_type=self.content_type())
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/client.py", line 301, in do_request
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self.request(url, method, **kwargs)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/client.py", line 296, in request
2015-03-29 06:50:05.244 TRACE nova.api.openstack     resp = super(SessionClient, self).request(*args, **kwargs)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in request
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self.session.request(url, method, **kwargs)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
2015-03-29 06:50:05.244 TRACE nova.api.openstack     return func(*args, **kwargs)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 382, in request
2015-03-29 06:50:05.244 TRACE nova.api.openstack     resp = send(**kwargs)
2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 426, in _send_request
2015-03-29 06:50:05.244 TRACE nova.api.openstack     raise exceptions.ConnectionRefused(msg)
2015-03-29 06:50:05.244 TRACE nova.api.openstack ConnectionRefused: Unable to establish connection to http://10.0.0.40:9696/v2.0/ports.json?device_id=be9cd6ae-091e-47ba-b3f8-4139b6326df2
2015-03-29 06:50:05.244 TRACE nova.api.openstack

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1437863

Title:
  API: doing "nova list" when neutron service is down has an internal
  error

Status in OpenStack Compute (Nova):
  New

Bug description:
  nicira@Ubuntu1204Server:~$ nova list
  ERROR (ClientException): The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-014bcadf-317f-4d2f-b0a0-dbebfc90bedf)

  The trace from the API is below

  2015-03-29 06:50:05.244 ERROR nova.api.openstack [req-014bcadf-317f-4d2f-b0a0-dbebfc90bedf demo demo] Caught error: Unable to establish connection to http://10.0.0.40:9696/v2.0/ports.json?device_id=be9cd6ae-091e-47ba-b3f8-4139b6326df2
  2015-03-29 06:50:05.244 TRACE nova.api.openstack Traceback (most recent call last):
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/__init__.py", line 125, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return req.get_response(self.application)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     application, catch_exc_info=False)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return resp(environ, start_response)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 634, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self._call_app(env, start_response)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 554, in _call_app
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self._app(env, _fake_start_response)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return resp(environ, start_response)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return resp(environ, start_response)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 136, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     response = self.app(environ, start_response)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return resp(environ, start_response)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 756, in __call__
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     content_type, body, accept)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 847, in _process_stack
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     request, action_args)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 710, in post_process_extensions
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     **action_args)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/security_groups.py", line 471, in detail
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     self._extend_servers(req, list(resp_obj.obj['servers']))
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/security_groups.py", line 422, in _extend_servers
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     servers))
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/security_group/neutron_driver.py", line 353, in get_instances_security_groups_bindings
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     ports = self._get_ports_from_server_list(servers, neutron)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/security_group/neutron_driver.py", line 312, in _get_ports_from_server_list
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     ports.extend(neutron.list_ports(**search_opts).get('ports'))
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 99, in with_params
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     ret = self.function(instance, *args, **kwargs)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 497, in list_ports
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     **_params)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 304, in list
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     for r in self._pagination(collection, path, **params):
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 317, in _pagination
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     res = self.get(path, params=params)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 290, in get
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     headers=headers, params=params)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 267, in retry_request
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     headers=headers, params=params)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 197, in do_request
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     content_type=self.content_type())
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/client.py", line 301, in do_request
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self.request(url, method, **kwargs)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/neutronclient/client.py", line 296, in request
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     resp = super(SessionClient, self).request(*args, **kwargs)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in request
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return self.session.request(url, method, **kwargs)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     return func(*args, **kwargs)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 382, in request
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     resp = send(**kwargs)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 426, in _send_request
  2015-03-29 06:50:05.244 TRACE nova.api.openstack     raise exceptions.ConnectionRefused(msg)
  2015-03-29 06:50:05.244 TRACE nova.api.openstack ConnectionRefused: Unable to establish connection to http://10.0.0.40:9696/v2.0/ports.json?device_id=be9cd6ae-091e-47ba-b3f8-4139b6326df2
  2015-03-29 06:50:05.244 TRACE nova.api.openstack

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1437863/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next