← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1291364] [NEW] _destroy_evacuated_instances fails randomly with high number of deleted instances

 

Public bug reported:

In our production environment (2013.2.1), we're facing a random error
thrown while starting nova-compute in Hyper-V nodes.

The following exception is thrown while calling
'_destroy_evacuated_instances':

16:30:58.802 7248 ERROR nova.openstack.common.threadgroup [-] 'NoneType' object is not iterable
2014-03-05 16:30:58.802 7248 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
(...)
2014-03-05 16:30:58.802 7248 TRACE nova.openstack.common.threadgroup   File "C:\Python27\lib\site-packages\nova\compute\manager.py", line 532, in _get_instances_on_driver
2014-03-05 16:30:58.802 7248 TRACE nova.openstack.common.threadgroup     name_map = dict((instance['name'], instance) for instance in instances)
2014-03-05 16:30:58.802 7248 TRACE nova.openstack.common.threadgroup TypeError: 'NoneType' object is not iterable

Full trace: http://paste.openstack.org/show/73243/

Our first guess is that this problem is related with number of deleted
instances in our deployment (~3000), they're all fetched in order to
check evacuated instances (as Hyper-V is not implementing
"list_instance_uuids").

In the case of KVM, this error is not happening as it's using a smarter
method to get this list based on the UUID of the instances.

Although this is being reported using Hyper-V, it's a problem that could
occur in other drivers not implementing "list_instance_uuids"

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: compute conductor hyper-v

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1291364

Title:
  _destroy_evacuated_instances fails randomly with high number of
  deleted instances

Status in OpenStack Compute (Nova):
  New

Bug description:
  In our production environment (2013.2.1), we're facing a random error
  thrown while starting nova-compute in Hyper-V nodes.

  The following exception is thrown while calling
  '_destroy_evacuated_instances':

  16:30:58.802 7248 ERROR nova.openstack.common.threadgroup [-] 'NoneType' object is not iterable
  2014-03-05 16:30:58.802 7248 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
  (...)
  2014-03-05 16:30:58.802 7248 TRACE nova.openstack.common.threadgroup   File "C:\Python27\lib\site-packages\nova\compute\manager.py", line 532, in _get_instances_on_driver
  2014-03-05 16:30:58.802 7248 TRACE nova.openstack.common.threadgroup     name_map = dict((instance['name'], instance) for instance in instances)
  2014-03-05 16:30:58.802 7248 TRACE nova.openstack.common.threadgroup TypeError: 'NoneType' object is not iterable

  Full trace: http://paste.openstack.org/show/73243/

  Our first guess is that this problem is related with number of deleted
  instances in our deployment (~3000), they're all fetched in order to
  check evacuated instances (as Hyper-V is not implementing
  "list_instance_uuids").

  In the case of KVM, this error is not happening as it's using a
  smarter method to get this list based on the UUID of the instances.

  Although this is being reported using Hyper-V, it's a problem that
  could occur in other drivers not implementing "list_instance_uuids"

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1291364/+subscriptions


Follow ups

References