yahoo-eng-team team mailing list archive

[OpenStack Infra <1806064@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/621692
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6d0386058b9628bbfcf64abdd707ad87ee19353c
Submitter: Zuul
Branch:    master

commit 6d0386058b9628bbfcf64abdd707ad87ee19353c
Author: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date:   Mon Dec 3 15:12:12 2018 -0500

    Create BDMs/tags in cell with instance when over-quota
    
    If the server create build request fails the quota check
    after the instance record has been created in a cell, we also
    need to create the BDMs and tags in that cell so that users
    can still see the tags on the server and so the API can
    properly cleanup volume attachments when the server is deleted.
    
    This change updates _cleanup_build_artifacts to create BDMs
    and tags in the same cell as the instance prior to deleting the
    build request and request spec and adjusts the assertions in the
    related functional test to show the bug is fixed.
    
    As for instances that get buried in cell0 due to scheduling
    failures, the tags are not created there so comments are left
    in those code paths to fix that issue as well, but that can be
    done separately from this patch.
    
    Change-Id: I1a9bdb596f74605ab4613c9cb2574e976aebbd8c
    Closes-Bug: #1806064


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1806064

Title:
  Volume remains in attaching/reserved status, if the instance is
  deleted after TooManyInstances exception in nova-conductor

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) pike series:
  Triaged
Status in OpenStack Compute (nova) queens series:
  Triaged
Status in OpenStack Compute (nova) rocky series:
  Triaged

Bug description:
  If a number of instances are booted from volumes in parallel and some
  of the build requests failed in nova-conductor with exception
  TooManyInstances [1] because of the setting quota.recheck_quota=True
  being set in nova.conf, some instances will end up in the ERROR state.

  If we delete this instances, their volumes will remain in
  attaching(Pike)/reserved(Queens) state.

  This bug is related to https://bugs.launchpad.net/nova/+bug/1404867

  Steps to reproduce:

  0. Set quota.recheck_quota=True, start several nova-conductors.

  1. Set VCPU quota limits for the project to 1.

  2. Create two instances with 1 VCPU in parallel.

  3. One of this instances will be created and one will end up in the
  ERROR state. Or both of them will be in ERROR state.

  4. Delete instances.

  5. Volumes from errored instances will not be available, they can't be
  attached, they can't be deleted without permision in
  volume:force_delete cinder policy.

  This bug exists at least in Pike (7ff1b28) and Queens (c5fe051).

  ---
  [1] https://github.com/openstack/nova/blob/stable/rocky/nova/conductor/manager.py#L1308

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1806064/+subscriptions