yahoo-eng-team team mailing list archive

[Alex Xu <soulxu@xxxxxxxxx>]

Public bug reported:

When instance was deleted in the db, but it is still at compute node,
the resource tracker will fail to update available resource.


2016-03-04 10:58:28.143 ERROR nova.compute.manager [req-d2f1c99b-0e81-4b6d-9361-a40bd2218141 None None] Error updating resources for node vm6.


2016-03-04 10:58:28.143 TRACE nova.compute.manager Traceback (most recent call last):
2016-03-04 10:58:28.143 TRACE nova.compute.manager   File "/opt/stack/nova/nova/compute/manager.py", line 6379, in update_available_resource
2016-03-04 10:58:28.143 TRACE nova.compute.manager     rt.update_available_resource(context)
2016-03-04 10:58:28.143 TRACE nova.compute.manager   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 491, in update_available_resource
2016-03-04 10:58:28.143 TRACE nova.compute.manager     resources = self.driver.get_available_resource(self.nodename)
2016-03-04 10:58:28.143 TRACE nova.compute.manager   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5414, in get_available_resource
2016-03-04 10:58:28.143 TRACE nova.compute.manager     disk_over_committed = self._get_disk_over_committed_size_total()
2016-03-04 10:58:28.143 TRACE nova.compute.manager   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7047, in _get_disk_over_committed_size_total
2016-03-04 10:58:28.143 TRACE nova.compute.manager     local_instances[guest.uuid], bdms[guest.uuid])
2016-03-04 10:58:28.143 TRACE nova.compute.manager KeyError: '49505c88-b38a-4100-ab56-97958b48b533'
2016-03-04 10:58:28.143 TRACE nova.compute.manager


The available resource won't get update until periodic_task
'_cleanup_running_deleted_instances' if running_deleted_instance_action
is 'reap'

** Affects: nova
     Importance: Undecided
     Assignee: Alex Xu (xuhj)
         Status: New

** Changed in: nova
     Assignee: (unassigned) => Alex Xu (xuhj)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1553144

Title:
  When some instance aren't deleted correctly and libvirt still keep the
  domain for the instance, the resource tracker will failed to update
  available resource

Status in OpenStack Compute (nova):
  New

Bug description:
  When instance was deleted in the db, but it is still at compute node,
  the resource tracker will fail to update available resource.

  
  2016-03-04 10:58:28.143 ERROR nova.compute.manager [req-d2f1c99b-0e81-4b6d-9361-a40bd2218141 None None] Error updating resources for node vm6.

  
  2016-03-04 10:58:28.143 TRACE nova.compute.manager Traceback (most recent call last):
  2016-03-04 10:58:28.143 TRACE nova.compute.manager   File "/opt/stack/nova/nova/compute/manager.py", line 6379, in update_available_resource
  2016-03-04 10:58:28.143 TRACE nova.compute.manager     rt.update_available_resource(context)
  2016-03-04 10:58:28.143 TRACE nova.compute.manager   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 491, in update_available_resource
  2016-03-04 10:58:28.143 TRACE nova.compute.manager     resources = self.driver.get_available_resource(self.nodename)
  2016-03-04 10:58:28.143 TRACE nova.compute.manager   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5414, in get_available_resource
  2016-03-04 10:58:28.143 TRACE nova.compute.manager     disk_over_committed = self._get_disk_over_committed_size_total()
  2016-03-04 10:58:28.143 TRACE nova.compute.manager   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7047, in _get_disk_over_committed_size_total
  2016-03-04 10:58:28.143 TRACE nova.compute.manager     local_instances[guest.uuid], bdms[guest.uuid])
  2016-03-04 10:58:28.143 TRACE nova.compute.manager KeyError: '49505c88-b38a-4100-ab56-97958b48b533'
  2016-03-04 10:58:28.143 TRACE nova.compute.manager


  The available resource won't get update until periodic_task
  '_cleanup_running_deleted_instances' if
  running_deleted_instance_action is 'reap'

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1553144/+subscriptions