← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1978983] Re: evacuate is not possible if the instance has task_state

 

Reviewed:  https://review.opendev.org/c/openstack/nova/+/848886
Committed: https://opendev.org/openstack/nova/commit/db919aa15f24c0d74f3c5c0e8341fad3f2392e57
Submitter: "Zuul (22348)"
Branch:    master

commit db919aa15f24c0d74f3c5c0e8341fad3f2392e57
Author: Amit Uniyal <auniyal@xxxxxxxxxx>
Date:   Wed Jul 6 18:20:02 2022 +0000

    For evacuation, ignore if task_state is not None
    
    ignore instance task state and continue with vm evacutaion
    
    Closes-Bug: #1978983
    Change-Id: I5540df6c7497956219c06cff6f15b51c2c8bc29d


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1978983

Title:
  evacuate is not possible if the instance has task_state

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) train series:
  Triaged
Status in OpenStack Compute (nova) ussuri series:
  Triaged
Status in OpenStack Compute (nova) victoria series:
  Triaged
Status in OpenStack Compute (nova) wallaby series:
  Triaged
Status in OpenStack Compute (nova) xena series:
  Triaged
Status in OpenStack Compute (nova) yoga series:
  Triaged

Bug description:
  Description
  ===========
  A compute host dies but before anything notices it a VM that was running on that host is requested to be stopped by the user. The VM task_state is set to powering-off and the shutdown RPC is sent to the dead compute. A bit later the monitoring system detect that the compute is dead and fences the compute, set the compute to forced_down in nova and triggers the evacuation of the VM. However the evacuation is rejected by nova:

  Cannot 'evacuate' instance 81451eb2-4600-4036-a6f1-b99139f0d277 while
  it is in task_state powering-off (HTTP 409) (Request-ID:
  req-363ca0a3-0d68-42f6-95d2-122bd2a53463)


  Steps to reproduce
  ==================
  0) deploy a multi node devstack
  1) create a VM
     $openstack --os-compute-api-version 2.80 server create --image cirros-0.5.2-x86_64-disk --flavor c1 --nic net-id=public  --use-config-drive vm1 --wait
  2) stop the nova-compute service of the host the VM is scheduled to:
     $sudo systemctl stop devstack@n-cpu
  3) stop the VM
     $openstack server stop vm1
  4) fence the host and force the host down in nova
  5) try to evacuate the VM
     $server evacuate vm1

  See also [1]

  Expected result
  ===============
  The VM is evacuated successfully 

  Actual result
  =============
  Cannot 'evacuate' instance 81451eb2-4600-4036-a6f1-b99139f0d277 while it is in task_state powering-off (HTTP 409) (Request-ID: req-363ca0a3-0d68-42f6-95d2-122bd2a53463)

  Environment
  ===========
  devstack on recent master

  Workaround
  ==========
  The admin can reset the state of the VM with 
  $nova reset-state --active vm1 
  then retry the evacuation.

  [1] https://paste.opendev.org/show/bQphEfOf8eLBnM6XmleQ/
  [2] https://paste.opendev.org/show/bVI7D8H5g9Oqjjo4rKfk/

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1978983/+subscriptions



References