yahoo-eng-team team mailing list archive

[Lee Yarwood <lyarwood@xxxxxxxxxx>]

Public bug reported:

Description
===========

I'm not entirely convinced that this is a bug but wanted to document and
discuss this upstream.

When using the rbd imagebackend, snapshots used to shelve an instance
cannot be removed after unshelving as they are cloned and as a result
are now the parents of the recreated instance disks.

This is in line with the behaviour of the imagebackend when initially
spawning an instance from an image but has caused confusion for
operators downstream who assume that the snapshot can be removed once
the instance has been unshelved.

We could flatten the instance disk when spawning during an unshelve but
to do so would mean extending the imagebackend to handle yet another
corner case for rbd.

Steps to reproduce
==================

$ nova boot --image cirros-raw --flavor 1 test-shelve
[..]
$ nova shelve test-shelve
[..]
$ nova unshelve test-shelve
[..]
$ sudo rbd -p vms ls -l
NAME                                       SIZE PARENT                                           FMT PROT LOCK 
4c843671-879d-4ba6-b4e8-8eefdced5393_disk 1024M images/df96af36-5a97-4f47-a79f-f3f3c85a21d9@snap   2 
$ glance image-delete df96af36-5a97-4f47-a79f-f3f3c85a21d9
Unable to delete image 'df96af36-5a97-4f47-a79f-f3f3c85a21d9' because it is in use.

We can easily workaround this by manually flattening the instance disk :

$ nova stop test-shelve
$ sudo rbd -p vms flatten 4c843671-879d-4ba6-b4e8-8eefdced5393_disk
Image flatten: 100% complete...done.
$ nova start test-shelve
$ glance image-delete df96af36-5a97-4f47-a79f-f3f3c85a21d9

Expected result
===============
Able to remove the shelved snapshot from Glance after unshelve.

Actual result
=============
Unable to remove the shelved snapshot from Glance after unshelve.

Environment
===========
1. Exact version of OpenStack you are running. See the following
   list for all releases: http://docs.openstack.org/releases/

   $ pwd
   /opt/stack/nova
   $ git rev-parse HEAD
   d768bfa2c2fb774154a5268f58b28537f7b39f69
   
2. Which hypervisor did you use?
   (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
   What's the version of that?

   libvirt + kvm

2. Which storage type did you use?
   (For example: Ceph, LVM, GPFS, ...)
   What's the version of that?

   ceph

3. Which networking type did you use?
   (For example: nova-network, Neutron with OpenVSwitch, ...)

   N/A

Logs & Configs
==============

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1653953

Title:
  Unable to remove snapshots after an instance is unshelved when using
  the rbd imagebackend

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========

  I'm not entirely convinced that this is a bug but wanted to document
  and discuss this upstream.

  When using the rbd imagebackend, snapshots used to shelve an instance
  cannot be removed after unshelving as they are cloned and as a result
  are now the parents of the recreated instance disks.

  This is in line with the behaviour of the imagebackend when initially
  spawning an instance from an image but has caused confusion for
  operators downstream who assume that the snapshot can be removed once
  the instance has been unshelved.

  We could flatten the instance disk when spawning during an unshelve
  but to do so would mean extending the imagebackend to handle yet
  another corner case for rbd.

  Steps to reproduce
  ==================

  $ nova boot --image cirros-raw --flavor 1 test-shelve
  [..]
  $ nova shelve test-shelve
  [..]
  $ nova unshelve test-shelve
  [..]
  $ sudo rbd -p vms ls -l
  NAME                                       SIZE PARENT                                           FMT PROT LOCK 
  4c843671-879d-4ba6-b4e8-8eefdced5393_disk 1024M images/df96af36-5a97-4f47-a79f-f3f3c85a21d9@snap   2 
  $ glance image-delete df96af36-5a97-4f47-a79f-f3f3c85a21d9
  Unable to delete image 'df96af36-5a97-4f47-a79f-f3f3c85a21d9' because it is in use.

  We can easily workaround this by manually flattening the instance disk
  :

  $ nova stop test-shelve
  $ sudo rbd -p vms flatten 4c843671-879d-4ba6-b4e8-8eefdced5393_disk
  Image flatten: 100% complete...done.
  $ nova start test-shelve
  $ glance image-delete df96af36-5a97-4f47-a79f-f3f3c85a21d9

  Expected result
  ===============
  Able to remove the shelved snapshot from Glance after unshelve.

  Actual result
  =============
  Unable to remove the shelved snapshot from Glance after unshelve.

  Environment
  ===========
  1. Exact version of OpenStack you are running. See the following
     list for all releases: http://docs.openstack.org/releases/

     $ pwd
     /opt/stack/nova
     $ git rev-parse HEAD
     d768bfa2c2fb774154a5268f58b28537f7b39f69
     
  2. Which hypervisor did you use?
     (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
     What's the version of that?

     libvirt + kvm

  2. Which storage type did you use?
     (For example: Ceph, LVM, GPFS, ...)
     What's the version of that?

     ceph

  3. Which networking type did you use?
     (For example: nova-network, Neutron with OpenVSwitch, ...)

     N/A

  Logs & Configs
  ==============

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1653953/+subscriptions