yahoo-eng-team team mailing list archive

[Sang Tran <2048154@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Hi Community,

I've got the error with this patch when rebooting vm with volume source
from the image, the result is Error with VolumeDeviceNotFound (Tested
with Netapp & PowerStrore iSCSI SAN driver)

Environment:
OpenStack Bobcat Stable version
Cinder driver: Tested with both Netapp & Powerstore iSCSI driver

Reproduce:
- Create VM from image (volume source type image, des type volume)
- Hard reboot or soft reboot
- VM damage with state Error, the volume attachment from cinder.volume_attachment has been deleted
- The LUN mapping has been remove from the SAN storage (cause of volume_attachment record is deleted)

Workaround:
- Recover the attachment record by setting the column deleted to 0
- Manually set the LUN mapping on SAN storage to the corresponding LUN id from attachment record
- Hard reboot and VM running

Reference;
[1] https://review.opendev.org/c/openstack/nova/+/882284

        for bdm in bdms.objects:
            if bdm.volume_id and bdm.source_type == 'volume' and \ ==> This line lead to bug
                bdm.destination_type == 'volume':
                try:
                    self.volume_api.attachment_get(context, bdm.attachment_id)
                except exception.VolumeAttachmentNotFound:
                    LOG.info(
                        f"Removing stale volume attachment "
                        f"'{bdm.attachment_id}' from instance for "
                        f"volume '{bdm.volume_id}'.", instance=instance)
                    bdm.destroy()
                    bdms_to_delete.append(bdm)
                else:
                    nova_attachments.append(bdm.attachment_id)

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2048154

Title:
  Vm reboot to Error state cause delete dangling bdms

Status in OpenStack Compute (nova):
  New

Bug description:
  Hi Community,

  I've got the error with this patch when rebooting vm with volume
  source from the image, the result is Error with VolumeDeviceNotFound
  (Tested with Netapp & PowerStrore iSCSI SAN driver)

  Environment:
  OpenStack Bobcat Stable version
  Cinder driver: Tested with both Netapp & Powerstore iSCSI driver

  Reproduce:
  - Create VM from image (volume source type image, des type volume)
  - Hard reboot or soft reboot
  - VM damage with state Error, the volume attachment from cinder.volume_attachment has been deleted
  - The LUN mapping has been remove from the SAN storage (cause of volume_attachment record is deleted)

  Workaround:
  - Recover the attachment record by setting the column deleted to 0
  - Manually set the LUN mapping on SAN storage to the corresponding LUN id from attachment record
  - Hard reboot and VM running

  Reference;
  [1] https://review.opendev.org/c/openstack/nova/+/882284

          for bdm in bdms.objects:
              if bdm.volume_id and bdm.source_type == 'volume' and \ ==> This line lead to bug
                  bdm.destination_type == 'volume':
                  try:
                      self.volume_api.attachment_get(context, bdm.attachment_id)
                  except exception.VolumeAttachmentNotFound:
                      LOG.info(
                          f"Removing stale volume attachment "
                          f"'{bdm.attachment_id}' from instance for "
                          f"volume '{bdm.volume_id}'.", instance=instance)
                      bdm.destroy()
                      bdms_to_delete.append(bdm)
                  else:
                      nova_attachments.append(bdm.attachment_id)

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2048154/+subscriptions