yahoo-eng-team team mailing list archive

[Tobias Urdin <2129498@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When a reimage of a BFV instance fails, let's say because it's trying to
use a image snapshot and the cinder reimage of the volume fails [2], the
boot volume gets status=error and since we've have nuked the volume
attachments prior to calling Cinder the volume attachments for the
volume is non-existent.

We can issue a hard reboot on the instance and it will boot but the boot
volume will be broken and live migrations can fail because volume
attachments in cinder is empty and status is error.

The only way to restore the correct volume attachment is:

- Power off instance (optional)
- Hack the deleted volume_attachment directly in the Cinder DB back and set the reserved volume attachment to deleted
- Set volume back to "in-use" status and attach_status "attached"
- Hard reboot instance (if you powered it off)

This is really bad, wouldn't it make more sense that we revert back to
the old volume attachment if the reimage fails?

Related bugs:
[1] https://bugs.launchpad.net/nova/+bug/2129468
[2] https://bugs.launchpad.net/cinder/+bug/2062539

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2129498

Title:
  failed reimage of BFV instance leaves volume in error with broken
  attachments

Status in OpenStack Compute (nova):
  New

Bug description:
  When a reimage of a BFV instance fails, let's say because it's trying
  to use a image snapshot and the cinder reimage of the volume fails
  [2], the boot volume gets status=error and since we've have nuked the
  volume attachments prior to calling Cinder the volume attachments for
  the volume is non-existent.

  We can issue a hard reboot on the instance and it will boot but the
  boot volume will be broken and live migrations can fail because volume
  attachments in cinder is empty and status is error.

  The only way to restore the correct volume attachment is:

  - Power off instance (optional)
  - Hack the deleted volume_attachment directly in the Cinder DB back and set the reserved volume attachment to deleted
  - Set volume back to "in-use" status and attach_status "attached"
  - Hard reboot instance (if you powered it off)

  This is really bad, wouldn't it make more sense that we revert back to
  the old volume attachment if the reimage fails?

  Related bugs:
  [1] https://bugs.launchpad.net/nova/+bug/2129468
  [2] https://bugs.launchpad.net/cinder/+bug/2062539

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2129498/+subscriptions