yahoo-eng-team team mailing list archive

[Tyler Blakeslee <tblakes@xxxxxxxxxx>]

Public bug reported:

Description:
When attempting to deploy an instance, if the instance fails to deploy on multiple hosts, the volumes are not cleaned up after the last deploy fails. The issue appears to only affect instances which fail an initial deploy and are rescheduled, and still ultimately fail for either NoValidHost or MaxRetriesExceeded. Instances which fail the initial deploy are cleaned up correctly.

Steps to Reproduce:
 - Perform a virtual machine deploy that fails an initial deploy and attempts to reschedule.
 - Raise an error during the reschedule.

Expected Result:
 - Volumes should be cleaned up similarly to when exception are raised in nova.compute.manager.ComputeManager._build_and_run_instance().

Actual Result:
 - Volumes are not cleaned up and remain even after the virtual machine deploy fails.

Additional Notes:
In nova.conductor.manager.ComputeTaskManager.build_instances(), an exception can be raised by the call to self._schedule_instances(). If this occurs, the exception is caught, but only the networks are cleaned up. No clean up of the volumes is performed.
 - https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L551-L565
In nova.compute.manager.ComputeManager._do_build_and_run_instance(), if an exception is raised by the call to self._build_and_run_instance() the volumes are cleaned up by a call to self._cleanup_volumes()
 - https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1854-L1866

Environment:
 - nova version: openstack-nova-16.0.0

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1730761

Title:
  Volumes not cleaned up on rescheduled VM deploys

Status in OpenStack Compute (nova):
  New

Bug description:
  Description:
  When attempting to deploy an instance, if the instance fails to deploy on multiple hosts, the volumes are not cleaned up after the last deploy fails. The issue appears to only affect instances which fail an initial deploy and are rescheduled, and still ultimately fail for either NoValidHost or MaxRetriesExceeded. Instances which fail the initial deploy are cleaned up correctly.

  Steps to Reproduce:
   - Perform a virtual machine deploy that fails an initial deploy and attempts to reschedule.
   - Raise an error during the reschedule.

  Expected Result:
   - Volumes should be cleaned up similarly to when exception are raised in nova.compute.manager.ComputeManager._build_and_run_instance().

  Actual Result:
   - Volumes are not cleaned up and remain even after the virtual machine deploy fails.

  Additional Notes:
  In nova.conductor.manager.ComputeTaskManager.build_instances(), an exception can be raised by the call to self._schedule_instances(). If this occurs, the exception is caught, but only the networks are cleaned up. No clean up of the volumes is performed.
   - https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L551-L565
  In nova.compute.manager.ComputeManager._do_build_and_run_instance(), if an exception is raised by the call to self._build_and_run_instance() the volumes are cleaned up by a call to self._cleanup_volumes()
   - https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1854-L1866

  Environment:
   - nova version: openstack-nova-16.0.0

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1730761/+subscriptions