yahoo-eng-team team mailing list archive

[OpenStack Infra <1569641@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/304929
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=5674e7646d106751b27d191e3334d9e6ebe9ab1b
Submitter: Jenkins
Branch:    master

commit 5674e7646d106751b27d191e3334d9e6ebe9ab1b
Author: Matt Riedemann <mriedem@xxxxxxxxxx>
Date:   Tue Apr 12 22:09:16 2016 -0400

    Properly clean up BDMs when _provision_instances fails
    
    _provision_instances calls create_db_entry_for_new_instance
    which creates the instance and block device mappings in the
    database.
    
    The instance is added to the instances list which is used
    in the global exception block at the bottom of _provision_instances
    to destroy any instances created. A failure that can trigger
    this cleanup attempt after the instance and BDMs are created
    is when checking server group member count fails with OverQuota.
    
    The problem is that we fail to (soft) delete the block device mappings
    that we created. Since there is a foreign key constraint between
    the block_device_mapping and instances tables in the database,
    when we try to archive (copy soft deleted things to shadow tables
    and then hard-delete them) the deleted instance it will fail on
    a referential constraint error due to the BDM(s) which weren't deleted.
    
    We can fix this by deleting the BDMs when deleting the instance just
    like we do for other reference tables.
    
    A functional test is added to demonstrate the failure and fix which
    also has the nice benefit of functionally testing the server group
    member overquota error handling.
    
    Change-Id: Ida66a93031046bafcf30c95ca232fb6382c2597b
    Closes-Bug: #1569641


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1569641

Title:
  server group members are not deleted on failed server create overquota

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) liberty series:
  Confirmed
Status in OpenStack Compute (nova) mitaka series:
  Confirmed

Bug description:
  When creating instances in the database in the compute API, if we fail
  after creating them we attempt to delete the instances from the DB
  here:

  https://github.com/openstack/nova/blob/af7e83fef3bc2c005c581587e9230a4070f8feb9/nova/compute/api.py#L1033

  However, if there is a failure it's ignored and we continue and just
  re-raise the exception.

  The instances can fail to delete because of a referential constraint
  on the block device mappings created here:

  https://github.com/openstack/nova/blob/af7e83fef3bc2c005c581587e9230a4070f8feb9/nova/compute/api.py#L1471

  So if we don't delete those first, we can't cleanup the instances. You
  can recreate this by changing CONF.quota_server_group_members=0 and
  trying to boot a server into a server group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1569641/+subscriptions