yahoo-eng-team team mailing list archive

[Robert Hoffmann <2067279@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

If you delete a VM during it is live-migrated, its volume attachments
may not be deleted.

During a migration there are temporarily two attachments, one to the old
and one to the target host. If you delete the VM during the live-
migration, the original attachment remains, cell1.block_device_mapping
already contains the one on the new host (and is set to deleted) and the
corresponding one in cinder is set to deleted.

Can mostly be reproduced by:

1. Create a server with a volume attached,
   e.g. openstack server create --flavor xxx image yyy --network zzz --boot-from-volume 5 testserver

2. Check volume attachment:
   openstack volume attachment list --volume-id vvv --all-projects --os-volume-api-version 3.27

3. Migrate VM and delete it:
   openstack server migrate --live-migration testserver; openstack server delete testserver

4. Check volume attachment again:
   openstack volume attachment list --volume-id vvv --all-projects --os-volume-api-version 3.27

5. Check database entries in cell1.block_device_mapping:
   select * from block_device_mapping where volume_id = "vvv";

   and cinder.volume_attachment:
   select * from volume_attachment where volume_id = "vvv";


This is especially a problem in large environments where live migrations are triggered automatically by life cycle tools (and may take some time) and customers can delete their VMs at any time.

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: attachments live-migration volumes

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2067279

Title:
  Volume attachment is not deleted if a VM is deleted during a live
  migration

Status in OpenStack Compute (nova):
  New

Bug description:
  If you delete a VM during it is live-migrated, its volume attachments
  may not be deleted.

  During a migration there are temporarily two attachments, one to the
  old and one to the target host. If you delete the VM during the live-
  migration, the original attachment remains, cell1.block_device_mapping
  already contains the one on the new host (and is set to deleted) and
  the corresponding one in cinder is set to deleted.

  Can mostly be reproduced by:

  1. Create a server with a volume attached,
     e.g. openstack server create --flavor xxx image yyy --network zzz --boot-from-volume 5 testserver

  2. Check volume attachment:
     openstack volume attachment list --volume-id vvv --all-projects --os-volume-api-version 3.27

  3. Migrate VM and delete it:
     openstack server migrate --live-migration testserver; openstack server delete testserver

  4. Check volume attachment again:
     openstack volume attachment list --volume-id vvv --all-projects --os-volume-api-version 3.27

  5. Check database entries in cell1.block_device_mapping:
     select * from block_device_mapping where volume_id = "vvv";

     and cinder.volume_attachment:
     select * from volume_attachment where volume_id = "vvv";

  
  This is especially a problem in large environments where live migrations are triggered automatically by life cycle tools (and may take some time) and customers can delete their VMs at any time.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2067279/+subscriptions