yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1507493] [NEW] Adding a host to an aggregate will give a traceback if service is disabled

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Robert van Leeuwen <rovanleeuwen@xxxxxxxx>
Date: Mon, 19 Oct 2015 08:43:24 -0000
Reply-to: Bug 1507493 <1507493@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

When you disable the compute service of a host you cannot add it to an
aggregate.

To reproduce (tested on Kilo MOS packages on Ubuntu 14.04):

# nova-manage service disable compute01 nova-compute
# nova aggregate-add-host az1 compute01
ERROR (ClientException): The server has either erred or is incapable of performing the requested operation. (HTTP 500)

In the nova-api.log:

2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack Traceback (most recent call last):
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line 125, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return req.get_response(self.application)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     application, catch_exc_info=False)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return resp(environ, start_response)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 634, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return self._call_app(env, start_response)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 554, in _call_app
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return self._app(env, _fake_start_response)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return resp(environ, start_response)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return resp(environ, start_response)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     response = self.app(environ, start_response)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return resp(environ, start_response)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 756, in __call__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     content_type, body, accept)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 821, in _process_stack
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 911, in dispatch
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return method(req=request, **action_args)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/contrib/aggregates.py", line 178, in action
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return _actions[action](req, id, data)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/contrib/aggregates.py", line 51, in wrapped
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return fn(self, req, id, host, *args, **kwargs)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/contrib/aggregates.py", line 188, in _add_host
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     aggregate = self.api.add_host_to_aggregate(context, id, host)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     payload)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     six.reraise(self.type_, self.value, self.tb)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return f(self, context, *args, **kw)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 3679, in add_host_to_aggregate
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     aggregate=aggregate)
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 3597, in is_safe_to_update_az
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     host_az = host_azs.pop()
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack IndexError: pop from empty list
2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1507493

Title:
  Adding a host to an aggregate will give a traceback if service is
  disabled

Status in OpenStack Compute (nova):
  New

Bug description:
  When you disable the compute service of a host you cannot add it to an
  aggregate.

  To reproduce (tested on Kilo MOS packages on Ubuntu 14.04):

  # nova-manage service disable compute01 nova-compute
  # nova aggregate-add-host az1 compute01
  ERROR (ClientException): The server has either erred or is incapable of performing the requested operation. (HTTP 500)

  In the nova-api.log:

  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack Traceback (most recent call last):
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line 125, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return req.get_response(self.application)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     application, catch_exc_info=False)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return resp(environ, start_response)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 634, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return self._call_app(env, start_response)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 554, in _call_app
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return self._app(env, _fake_start_response)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return resp(environ, start_response)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return resp(environ, start_response)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     response = self.app(environ, start_response)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return resp(environ, start_response)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 756, in __call__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     content_type, body, accept)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 821, in _process_stack
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 911, in dispatch
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return method(req=request, **action_args)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/contrib/aggregates.py", line 178, in action
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return _actions[action](req, id, data)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/contrib/aggregates.py", line 51, in wrapped
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return fn(self, req, id, host, *args, **kwargs)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/contrib/aggregates.py", line 188, in _add_host
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     aggregate = self.api.add_host_to_aggregate(context, id, host)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     payload)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     six.reraise(self.type_, self.value, self.tb)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     return f(self, context, *args, **kw)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 3679, in add_host_to_aggregate
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     aggregate=aggregate)
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 3597, in is_safe_to_update_az
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack     host_az = host_azs.pop()
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack IndexError: pop from empty list
  2015-10-19 07:44:51.727 38561 TRACE nova.api.openstack

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1507493/+subscriptions
Follow ups

[Bug 1507493] Re: Adding a host to an aggregate will give a traceback if service is disabled
From: Matt Riedemann, 2015-11-02