yahoo-eng-team team mailing list archive

[melanie witt <1672624@xxxxxxxxxxxxxxxxxx>]

This behavior is expected in the scenario where nova-compute is brought
down but the instance domain is still running and keeping the volume
busy. Usually in practice, nova-compute should only be down in the case
of a failed compute host or one that has been properly fenced by the
operator. However, if nova-compute goes down temporarily and an instance
is local deleted, the instance domain will be reaped by the nova-compute
periodic task after nova-compute comes back up. After the domain is
reaped, the volume can be successfully deleted from cinder.

** Changed in: nova
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1672624

Title:
  Ceph volumes attached to local deleted instance could not be correctly
  handled

Status in Cinder:
  New
Status in OpenStack Compute (nova):
  Invalid

Bug description:
  How to reproduce:
  1. Launch an instance.
  2. Create a volume with ceph backend.
  3. Attach the volume created in step 3.
  4. Kill nova-compute
  5. Delete the instance, this will go to local_delete
  6. Check volumes status using "cinder list", the volume is in "available" status
  7. Try to delete the volume, failed:
  2017-03-14 11:40:41.050 DEBUG oslo_messaging._drivers.amqpdriver mreceived message with unique_id: 061b4f9b52aa425d97811c066133b170 from (pid=480) __call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:215 
  15:05 2017-03-14 11:40:41.056 DEBUG cinder.coordination req-774b4680-d861-4e16-bad4-a032ff0b3579  None  Lock "7c7d03d9-3244-4923-b72e-459677ee48aa-delete_volume" acquired by "delete_volume" :: waited 0.000s from (pid=480) _synchronized /opt/stack/cinder/cinder/coordination.py:300 
  15:05 2017-03-14 11:40:41.155 DEBUG cinder.volume.drivers.rbd req-774b4680-d861-4e16-bad4-a032ff0b3579  00;36madmin None connecting to ceph (timeout=-1). from (pid=480) _connect_to_rados /opt/stack/cinder/cinder/volume/drivers/rbd.py:299 
  15:05 2017-03-14 11:40:42.376 DEBUG cinder.volume.drivers.rbd req-774b4680-d861-4e16-bad4-a032ff0b3579   None volume has no backup snaps from (pid=480) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:660 
  15:05 2017-03-14 11:40:42.377 DEBUG cinder.volume.drivers.rbd req-774b4680-d861-4e16-bad4-a032ff0b3579 admin None Volume volume-7c7d03d9-3244-4923-b72e-459677ee48aa is not a clone.   from (pid=480) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:683 
  15:06 2017-03-14 11:40:42.382 DEBUG cinder.volume.drivers.rbd req-774b4680-d861-4e16-bad4-a032ff0b3579   None deleting rbd volume volume-7c7d03d9-3244-4923-b72e-459677ee48aa   from (pid=480) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:781 
  15:06 2017-03-14 11:40:42.570 DEBUG cinder.utils req-774b4680-d861-4e16-bad4-a032ff0b3579 admin None Failed attempt 1 from (pid=480) _print_stop /opt/stack/cinder/cinder/utils.py:780 
  ...
  15:07 2017-03-14 11:41:12.950 WARNING cinder.volume.drivers.rbd req-774b4680-d861-4e16-bad4-a032ff0b3579 ^admin NoneImageBusy error raised while deleting rbd volume. This may have been caused by a connection from a client that has crashed and, if so, may be resolved by retrying the delete after 30 seconds has elapsed.
  15:07 2017-03-14 11:41:12.955 ERROR cinder.volume.manager req-774b4680-d861-4e16-bad4-a032ff0b3579 admin None^Unable to delete busy volume.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1672624/+subscriptions