← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #52954

[Bug 1597234] [NEW] VM with encrypted volume goes to error state when hard reboot

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

In current master branch with LVM as backend:

Steps to reproduce
==================

1. cinder type-create LUKS

2.  cinder encryption-type-create --cipher aes-xts-plain64 --key_size
512   --control_location front-end LUKS
nova.volume.encryptors.luks.LuksEncryptor

3.  cinder create --volume-type LUKS 1

4.  nova boot --flavor 1 --image 3feb30f7-d171-4b58-a126-2127016a6051
lisa

5.  nova volume-attach c2ee07df-f1d2-4c1c-b08f-9d001209d4cf
72ce7ebf-7400-47da-91f5-3173e01a199e

6. nova reboot --hard c2ee07df-f1d2-4c1c-b08f-9d001209d4cf

Actual result
=============

The VM goes into error state.

2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server     device_info = self.connector.connect_volume(connection_info['data'])
2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner

....
2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server     cmd=sanitized_cmd)
2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server ProcessExecutionError: Unexpected error while running command.
2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server Command: sudo nova-rootwrap /etc/nova/rootwrap.conf scsi_id --page 0x83 --whitelisted /dev/disk/by-path/ip-10.239.48.111:3260-iscsi-iqn.2010-10.org.openstack:volume-72ce7ebf-7400-47da-91f5-3173e01a199e-lun-1
2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server Exit code: 1
2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server Stdout: u''
2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server Stderr: u''
2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server

Analysis:

When attaching the encrypted volume to the VM, it finally makes the symlink path point to the dm device. 
When reboot, there is no unattach dm device. May problem is here. Need to investigate more.

-HP-Compaq-Elite-8300-CMT:/dev$ ls -lrta /dev/mapper/ip-10.239.48.111:3260-iscsi-iqn.2010-10.org.openstack:volume-72ce7ebf-7400-47da-91f5-3173e01a199e-lun-1
lrwxrwxrwx 1 root root 7 Jun 29 16:05 /dev/mapper/ip-10.239.48.111:3260-iscsi-iqn.2010-10.org.openstack:volume-72ce7ebf-7400-47da-91f5-3173e01a199e-lun-1 -> ../dm-2

** Affects: nova
     Importance: Undecided
     Assignee: Lisa Li (lisali)
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1597234

Title:
  VM with encrypted volume goes to error state when hard reboot

Status in OpenStack Compute (nova):
  New

Bug description:
  In current master branch with LVM as backend:

  Steps to reproduce
  ==================

  1. cinder type-create LUKS

  2.  cinder encryption-type-create --cipher aes-xts-plain64 --key_size
  512   --control_location front-end LUKS
  nova.volume.encryptors.luks.LuksEncryptor

  3.  cinder create --volume-type LUKS 1

  4.  nova boot --flavor 1 --image 3feb30f7-d171-4b58-a126-2127016a6051
  lisa

  5.  nova volume-attach c2ee07df-f1d2-4c1c-b08f-9d001209d4cf
  72ce7ebf-7400-47da-91f5-3173e01a199e

  6. nova reboot --hard c2ee07df-f1d2-4c1c-b08f-9d001209d4cf

  Actual result
  =============

  The VM goes into error state.

  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server     device_info = self.connector.connect_volume(connection_info['data'])
  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner

  ....
  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server     cmd=sanitized_cmd)
  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server ProcessExecutionError: Unexpected error while running command.
  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server Command: sudo nova-rootwrap /etc/nova/rootwrap.conf scsi_id --page 0x83 --whitelisted /dev/disk/by-path/ip-10.239.48.111:3260-iscsi-iqn.2010-10.org.openstack:volume-72ce7ebf-7400-47da-91f5-3173e01a199e-lun-1
  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server Exit code: 1
  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server Stdout: u''
  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server Stderr: u''
  2016-06-29 16:05:11.925 TRACE oslo_messaging.rpc.server

  Analysis:

  When attaching the encrypted volume to the VM, it finally makes the symlink path point to the dm device. 
  When reboot, there is no unattach dm device. May problem is here. Need to investigate more.

  -HP-Compaq-Elite-8300-CMT:/dev$ ls -lrta /dev/mapper/ip-10.239.48.111:3260-iscsi-iqn.2010-10.org.openstack:volume-72ce7ebf-7400-47da-91f5-3173e01a199e-lun-1
  lrwxrwxrwx 1 root root 7 Jun 29 16:05 /dev/mapper/ip-10.239.48.111:3260-iscsi-iqn.2010-10.org.openstack:volume-72ce7ebf-7400-47da-91f5-3173e01a199e-lun-1 -> ../dm-2

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1597234/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next