yahoo-eng-team team mailing list archive

[Robert Collins <1272623@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

This can happen if a deployment is interrupted at just the wrong time.

2014-01-25 06:53:38,781.781 14556 DEBUG nova.compute.manager [req-e1958f79-b0c0-4c80-b284-85bb56f1541d None None] [instance: e21e6bca-b528-4922-9f59-7a1a6534ec8d] Current state is 1, state in DB is 1. _init_instance /opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py:720
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 346, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
    result = function(*args, **kwargs)
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/openstack/common/service.py", line 480, in run_service
    service.start()
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/service.py", line 172, in start
    self.manager.init_host()
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 805, in init_host
    self._init_instance(context, instance)
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 684, in _init_instance
    self.driver.plug_vifs(instance, net_info)
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 538, in plug_vifs
    self._plug_vifs(instance, network_info)
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 543, in _plug_vifs
    node = _get_baremetal_node_by_instance_uuid(instance['uuid'])
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 85, in _get_baremetal_node_by_instance_uuid
    node = db.bm_node_get_by_instance_uuid(ctx, instance_uuid)
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/db/api.py", line 101, in bm_node_get_by_instance_uuid
    instance_uuid)
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 112, in wrapper
    return f(*args, **kwargs)
  File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/db/sqlalchemy/api.py", line 152, in bm_node_get_by_instance_uuid
    raise exception.InstanceNotFound(instance_id=instance_uuid)
InstanceNotFound: Instance 84c6090b-bf42-4c6a-b2ff-afb22b5ff156 could not be found.

If there is no allocated node, we can just skip that part of delete.

** Affects: nova
     Importance: High
         Status: Triaged


** Tags: baremetal

** Summary changed:

- nova refuses to delete baremetal instances if there is no associated node
+ nova refuses to start if there are baremetal instances with no associated node

** Description changed:

+ This can happen if a deployment is interrupted at just the wrong time.
+ 
  2014-01-25 06:53:38,781.781 14556 DEBUG nova.compute.manager [req-e1958f79-b0c0-4c80-b284-85bb56f1541d None None] [instance: e21e6bca-b528-4922-9f59-7a1a6534ec8d] Current state is 1, state in DB is 1. _init_instance /opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py:720
  Traceback (most recent call last):
-   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 346, in fire_timers
-     timer()
-   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 56, in __call__
-     cb(*args, **kw)
-   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
-     result = function(*args, **kwargs)
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/openstack/common/service.py", line 480, in run_service
-     service.start()
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/service.py", line 172, in start
-     self.manager.init_host()
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 805, in init_host
-     self._init_instance(context, instance)
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 684, in _init_instance
-     self.driver.plug_vifs(instance, net_info)
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 538, in plug_vifs
-     self._plug_vifs(instance, network_info)
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 543, in _plug_vifs
-     node = _get_baremetal_node_by_instance_uuid(instance['uuid'])
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 85, in _get_baremetal_node_by_instance_uuid
-     node = db.bm_node_get_by_instance_uuid(ctx, instance_uuid)
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/db/api.py", line 101, in bm_node_get_by_instance_uuid
-     instance_uuid)
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 112, in wrapper
-     return f(*args, **kwargs)
-   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/db/sqlalchemy/api.py", line 152, in bm_node_get_by_instance_uuid
-     raise exception.InstanceNotFound(instance_id=instance_uuid)
+   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 346, in fire_timers
+     timer()
+   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 56, in __call__
+     cb(*args, **kw)
+   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
+     result = function(*args, **kwargs)
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/openstack/common/service.py", line 480, in run_service
+     service.start()
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/service.py", line 172, in start
+     self.manager.init_host()
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 805, in init_host
+     self._init_instance(context, instance)
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 684, in _init_instance
+     self.driver.plug_vifs(instance, net_info)
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 538, in plug_vifs
+     self._plug_vifs(instance, network_info)
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 543, in _plug_vifs
+     node = _get_baremetal_node_by_instance_uuid(instance['uuid'])
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 85, in _get_baremetal_node_by_instance_uuid
+     node = db.bm_node_get_by_instance_uuid(ctx, instance_uuid)
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/db/api.py", line 101, in bm_node_get_by_instance_uuid
+     instance_uuid)
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 112, in wrapper
+     return f(*args, **kwargs)
+   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/db/sqlalchemy/api.py", line 152, in bm_node_get_by_instance_uuid
+     raise exception.InstanceNotFound(instance_id=instance_uuid)
  InstanceNotFound: Instance 84c6090b-bf42-4c6a-b2ff-afb22b5ff156 could not be found.
  
- 
  If there is no allocated node, we can just skip that part of delete.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1272623

Title:
  nova refuses to start if there are baremetal instances with no
  associated node

Status in OpenStack Compute (Nova):
  Triaged

Bug description:
  This can happen if a deployment is interrupted at just the wrong time.

  2014-01-25 06:53:38,781.781 14556 DEBUG nova.compute.manager [req-e1958f79-b0c0-4c80-b284-85bb56f1541d None None] [instance: e21e6bca-b528-4922-9f59-7a1a6534ec8d] Current state is 1, state in DB is 1. _init_instance /opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py:720
  Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 346, in fire_timers
      timer()
    File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 56, in __call__
      cb(*args, **kw)
    File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
      result = function(*args, **kwargs)
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/openstack/common/service.py", line 480, in run_service
      service.start()
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/service.py", line 172, in start
      self.manager.init_host()
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 805, in init_host
      self._init_instance(context, instance)
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 684, in _init_instance
      self.driver.plug_vifs(instance, net_info)
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 538, in plug_vifs
      self._plug_vifs(instance, network_info)
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 543, in _plug_vifs
      node = _get_baremetal_node_by_instance_uuid(instance['uuid'])
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 85, in _get_baremetal_node_by_instance_uuid
      node = db.bm_node_get_by_instance_uuid(ctx, instance_uuid)
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/db/api.py", line 101, in bm_node_get_by_instance_uuid
      instance_uuid)
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 112, in wrapper
      return f(*args, **kwargs)
    File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/db/sqlalchemy/api.py", line 152, in bm_node_get_by_instance_uuid
      raise exception.InstanceNotFound(instance_id=instance_uuid)
  InstanceNotFound: Instance 84c6090b-bf42-4c6a-b2ff-afb22b5ff156 could not be found.

  If there is no allocated node, we can just skip that part of delete.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1272623/+subscriptions