yahoo-eng-team team mailing list archive

[OpenStack Infra <1404867@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/335697
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=5ce74fa06c0e7a70fdc927b2c1f364af83f7de1d
Submitter: Jenkins
Branch:    master

commit 5ce74fa06c0e7a70fdc927b2c1f364af83f7de1d
Author: ankitagrawal <ankit11.agrawal@xxxxxxxxxxx>
Date:   Wed Sep 23 03:58:19 2015 -0700

    Detach volume after deleting instance with no host
    
    If an instance is booted from a volume, shelved, and goes into an error
    state due to some reason, the volume from which instance is booted
    remains even the instance is deleted because instance has no host
    associated with it.
    
    Called _local_delete() to detach volume and destroy bdm if instance is
    in shelved_offloaded state or has no host associated with it. This will
    cleanup both volumes and the networks.
    
    Note:
    Ankit had submitted same patch [1] earlier which was reverted [2] due
    to a race condition on jenkins if an instance is deleted when it is in
    building state.  The patch was then rebumitted [3] fixing the
    the failure of race condition by reverting the ObjectActionError
    exception handling in _delete.  This patch was later re-reverted [4]
    due to continued jenkins race conditions.
    
    The patch [5] intended to avoid the jenkins race condition by leaving
    the flow for instances in the BUILDING state unchanged and only calling
    _local_delete() on instances in the shelved_offloaded or error states
    when the instance has no host associated with it. It however also had
    to be reverted [6] because of yet another race condition.
    
    This version takes a more minimal approach of adding the ERROR state
    to the logic for doing a local delete plus cleanup of resources on
    a compute host. Comments have also been added to the existing code
    to explain more about the different flows.
    
    [1] Ic630ae7d026a9697afec46ac9ea40aea0f5b5ffb
    [2] Id4e405e7579530ed1c1f22ccc972d45b6d185f41
    [3] Ic107d8edc7ee7a4ebb04eac58ef0cdbf506d6173
    [4] Ibcbe35b5d329b183c4d0e8233e8ada26ebc512c2
    [5] I928a397c75b857e94bf5c002e50ec43a2bed9848
    [6] I6b9b886e0d6f2ec86141c048fb50969bccf5cb30
    
    Co-Authored-By: Ankit Agrawal <ankit11.agrawal@xxxxxxxxxxx>
    Co-Authored-By: Samuel Matzek <smatzek@xxxxxxxxxx>
    Co-Authored-By: melanie witt <melwittt@xxxxxxxxx>
    
    Closes-Bug: 1404867
    Closes-Bug: 1408527
    
    Change-Id: I2192ef513a2cd15d21e9d5d5fe22c5a5fbae0941


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1404867

Title:
  Volume remains in-use status, if instance booted from volume is
  deleted in error state

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  If the instance is booted from volume and goes in to error state due to some reason.
  Volume from which instance is booted, remains in-use state even the instance is deleted.
  IMO, volume should be detached so that it can be used to boot other instance.

  Steps to reproduce:

  1. Log in to Horizon, create a new volume.
  2. Create an Instance using newly created volume.
  3. Verify instance is in active state.
  $ source devstack/openrc demo demo
  $ nova list
  +--------------------------------------+------+--------+------------+-------------+------------------+
  | ID                                   | Name | Status | Task State | Power State | Networks         |
  +--------------------------------------+------+--------+------------+-------------+------------------+
  | dae3a13b-6aa8-4794-93cd-5ab7bf90f604 | nova | ACTIVE | -          | Running     | private=10.0.0.3 |
  +--------------------------------------+------+--------+------------+-------------+------------------+

  Note:
  Use shelve-unshelve api to see the instance goes into error state.
  unshelving volumed back instance does not work and sets instance state to error state (ref: https://bugs.launchpad.net/nova/+bug/1404801)

  4. Shelve the instance
  $ nova shelve <instance-uuid>

  5. Verify the status is SHELVED_OFFLOADED.
  $ nova list
  +--------------------------------------+------+-------------------+------------+-------------+------------------+
  | ID                                   | Name | Status            | Task State | Power State | Networks         |
  +--------------------------------------+------+-------------------+------------+-------------+------------------+
  | dae3a13b-6aa8-4794-93cd-5ab7bf90f604 | nova | SHELVED_OFFLOADED | -          | Shutdown    | private=10.0.0.3 |
  +--------------------------------------+------+-------------------+------------+-------------+------------------+

  6. Unshelve the instance.
  $ nova unshelve <instance-uuid>

  5. Verify the instance is in Error state.
  $ nova list
  +--------------------------------------+------+-------------------+------------+-------------+------------------+
  | ID                                   | Name | Status            | Task State | Power State | Networks         |
  +--------------------------------------+------+-------------------+------------+-------------+------------------+
  | dae3a13b-6aa8-4794-93cd-5ab7bf90f604 | nova | Error             | unshelving | Spawning    | private=10.0.0.3 |
  +--------------------------------------+------+-------------------+------------+-------------+------------------+

  6. Delete the instance using Horizon.

  7. Verify that volume still in in-use state
  $ cinder list
  +--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
  |                  ID                  | Status | Name | Size | Volume Type | Bootable |             Attached to              |
  +--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
  | 4aeefd25-10aa-42c2-9a2d-1c89a95b4d4f | in-use | test |  1   | lvmdriver-1 |   true   | 8f7bdc24-1891-4bbb-8f0c-732b9cbecae7 |
  +--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+

  8. In Horizon, volume "Attached To" information is displayed as
  "Attached to None on /dev/vda".

  9. User is not able to delete this volume, or attached it to another
  instance as it is still in use.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1404867/+subscriptions