yahoo-eng-team team mailing list archive

[Sylvain Bauza <sbauza@xxxxxxx>]

This is indeed fixed upstream, as you can see in the source code there
https://github.com/openstack/nova/blob/2cddf595a8cdedbdb844e800d853ea143817b545/nova/compute/manager.py#L721-L738

We only delete instances if the evacuation was either done, or just precreated.
If the migration wasn't good, then we don't delete the instance.


** Changed in: nova
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1636825

Title:
  Instances for which rebuild failed get deleted from source host

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Description
  ===========
  In the current implementation we have the method '_destroy_evacuated_instances' in compute.manager which deletes any instances from source after they have been evacuated. This method is called as part of host initialization (init_host) and checks the migration records for VMs which were evacuated.

  There is a possibility that if a VM fails as part of rebuild operation
  on destination host after creating a migration record, then when the
  source host is brought back up it may end up deleting the VM from
  source as well.

  To fix this we should check the 'host' attribute in instances table
  before deleting the VM and delete VM from source only if the host has
  been updated in db after rebuild.

  Steps to reproduce
  ==================
  * deploy a VM
  * Bring down the host where VM was deployed.
  * Evacuate the instance to another host where the rebuild operation may fail (insufficient resources or storage issue)
  * This will result in VM not being present on destination host.
  * Check that a migration record of type 'evacuation' is present in db.
  * Bring the source host up.

  Expected result
  ===============
  The VM should be present on the source host.

  Actual result
  =============
  VM gets deleted as part of evacuated instance cleanup on stat-up of compute service on source host.

  Environment
  ===========
  1. Exact version of OpenStack you are running. See the following
  Openstack Newton

  2. Which hypervisor did you use?
   PowerVM

  3. Which networking type did you use?
     Neutron with OpenVSwitch

  Logs & Configs
  ==============
  In the logs following message is seen on startup -
  2016-10-24 09:32:11.131 3169 INFO nova.compute.manager [req-6611fe85-0515-4cb4-b1c0-3f34f196a0c7 - - - - -] [instance: ed9ca4b9-8938-4d7b-9eec-1dd6ca7bc8c8] Deleting instance as it has been evacuated from this host

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1636825/+subscriptions