yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

Public bug reported:

This is purely based on code inspection right now, I need to write a
functional test to recreate the issue.

While triaging bug 1821594 it got me thinking about handling placement
allocations during resize when something fails, which got me thinking
about an older fix:

https://review.openstack.org/#/c/543971/6/nova/compute/manager.py@4457

Looking back on that now, I think the revert during resize_instance is
OK as long as the instance host/node has not changed, but I think doing
it when finish_resize fails was probably a mistake because the
instance.host in the nova db won't match where the allocations exist in
placement. Before Pike this was fine since the ResourceTracker would
heal the allocations in the update_available_resource periodic task, but
we don't have that anymore.

So this could result in an instance reported as being on the dest host
in the nova database with the new flavor, which is where it will get
rebuilt/rebooted/etc, but placement will be tracking the instance
resource allocations using the old flavor against the source host, which
is not where the instance is.

Furthermore, if finish_resize fails, the instance should be in ERROR
status and the user would likely try to hard reboot the instance to
correct that status, which would happen on the dest host.

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: placement resize

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1825537

Title:
  finish_resize failures incorrectly revert allocations

Status in OpenStack Compute (nova):
  New

Bug description:
  This is purely based on code inspection right now, I need to write a
  functional test to recreate the issue.

  While triaging bug 1821594 it got me thinking about handling placement
  allocations during resize when something fails, which got me thinking
  about an older fix:

  https://review.openstack.org/#/c/543971/6/nova/compute/manager.py@4457

  Looking back on that now, I think the revert during resize_instance is
  OK as long as the instance host/node has not changed, but I think
  doing it when finish_resize fails was probably a mistake because the
  instance.host in the nova db won't match where the allocations exist
  in placement. Before Pike this was fine since the ResourceTracker
  would heal the allocations in the update_available_resource periodic
  task, but we don't have that anymore.

  So this could result in an instance reported as being on the dest host
  in the nova database with the new flavor, which is where it will get
  rebuilt/rebooted/etc, but placement will be tracking the instance
  resource allocations using the old flavor against the source host,
  which is not where the instance is.

  Furthermore, if finish_resize fails, the instance should be in ERROR
  status and the user would likely try to hard reboot the instance to
  correct that status, which would happen on the dest host.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1825537/+subscriptions