yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #85946
[Bug 1925818] Re: Retype of non-encrypted root volume to encrypted fails
I've added cinder and os-brick here as AFAICT this isn't a Nova issue
given the description in c#0 and logs in c#1.
The attempt by Cinder to retype the detached volume doesn't appear to
have encrypted anything.
This might be the result of another issue in os-brick when connecting
the volume to the underlying host but that isn't obvious from the
compute logs.
Can you reproduce this with DEBUG logs on the compute so we can see what
os-brick is doing?
** Also affects: cinder
Importance: Undecided
Status: New
** Also affects: os-brick
Importance: Undecided
Status: New
** Changed in: nova
Status: New => Incomplete
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1925818
Title:
Retype of non-encrypted root volume to encrypted fails
Status in Cinder:
New
Status in OpenStack Compute (nova):
Incomplete
Status in os-brick:
New
Bug description:
Description
===========
I tried to retype volume from unencrypted to encrypted one. That works fine for non-root volumes, however it doesn't work when volume was/is root volume.
Steps to reproduce
==================
1. Create an instance with unencrypted root volume(select do not delete volume on termination)
2. Delete instance
3. Retype left volume from deleted instance to encrypted one and wait till retype will finish successfully
4. Launch an instance from this encrypted volume(increase original size by +2GB)
Expected result
===============
New instance is started from encrypted volume
Actual result
=============
Instance fails to start. At horizon we can see an error message:
Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 610, in build_instances filter_properties, instances[0].uuid) File "/usr/lib/python2.7/dist-packages/nova/scheduler/utils.py", line 680, in populate_retry raise exception.MaxRetriesExceeded(reason=msg) MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance d19504b4-8c05-423b-abaa-b1b8c0658fa8. Last exception: Unexpected error while running command. Command: cryptsetup --batch-mode luksFormat --key-file=- --cipher aes-xts-plain64 --key-size 256 /dev/dm-19 Exit code: 5 Stdout: u'' Stderr: u'Cannot use device /dev/dm-19 which is in use (already mapped or mounted).\n'
Environment
===========
1. Version of cinder:
ii cinder-api 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - API server
ii cinder-backup 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - Scheduler server
ii cinder-common 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - common files
ii cinder-scheduler 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - Scheduler server
ii cinder-volume 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - Volume server
ii python-cinder 2:13.0.9-0ubuntu1~cloud1.1 all Cinder Python 2 libraries
ii python-cinderclient 1:3.5.0-0ubuntu1 all Python bindings to the OpenStack Volume API - Python 2.x
Version of barbican:
ii barbican-api 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - API Server
ii barbican-common 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - common files
ii barbican-keystone-listener 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - Keystone Listener
ii barbican-worker 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - Worker Node
ii python-barbican 1:6.0.1-0ubuntu1 all OpenStack Key Management Service - Python files
ii python3-barbican 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - Python 3 files
ii python3-barbicanclient 4.10.0-0ubuntu1~cloud0 all OpenStack Key Management API client - Python 3.x
Version of nova:
ii nova-api 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - API frontend
ii nova-common 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - common files
ii nova-conductor 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - conductor service
ii nova-consoleauth 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - Console Authenticator
ii nova-novncproxy 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - NoVNC proxy
ii nova-placement-api 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - placement API frontend
ii nova-scheduler 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - virtual machine scheduler
ii python-nova 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute Python 2 libraries
ii python-novaclient 2:11.0.0-0ubuntu1~cloud0 all client library for OpenStack Compute API - Python 2.7
2. We use iSCSI storage(Huawei dorado)
Logs & Configs
==============
Cinder volume logs:
2021-04-23 16:33:43.288 17298 INFO cinder.volume.drivers.huawei.huawei_driver [req-19f3af30-022a-4c96-ab28-2c3aaa91764c 7d52a2a770e4402982390eb658425083 ab5b748ab5e04fa58c8b4aad9f744e2b - default default] Initialize iscsi connection for volume de2c38b7-bf67-42b7-ae51-76fca6c7c05b, connector info {u'initiator': u'iqn.1993-08.org.debian:01:3a7188e5b55f', u'ip': u'10.10.0.248', u'platform': u'x86_64', u'host': u'comp-c08', u'do_local_attach': False, u'mountpoint': u'/dev/vda', u'os_type': u'linux2', u'multipath': True}.
2021-04-23 16:34:05.883 17298 INFO cinder.volume.drivers.huawei.huawei_driver [req-969c4e48-91a7-4c46-824d-63b924503c34 7d52a2a770e4402982390eb658425083 ab5b748ab5e04fa58c8b4aad9f744e2b - default default] Terminate iscsi connection for volume de2c38b7-bf67-42b7-ae51-76fca6c7c05b, connector info {u'initiator': u'iqn.1993-08.org.debian:01:3a7188e5b55f', u'ip': u'10.10.0.248', u'platform': u'x86_64', u'host': u'comp-c08', u'do_local_attach': False, u'mountpoint': u'/dev/vda', u'os_type': u'linux2', u'multipath': True}.
2021-04-23 16:34:12.452 17298 INFO
cinder.volume.drivers.huawei.huawei_driver [req-314b1423-2650-4a06
-9f2e-945bd648e70f 7d52a2a770e4402982390eb658425083
ab5b748ab5e04fa58c8b4aad9f744e2b - default default] Initialize iscsi
connection for volume de2c38b7-bf67-42b7-ae51-76fca6c7c05b, connector
info {u'initiator': u'iqn.1993-08.org.debian:01:a1edf5552e8', u'ip':
u'10.10.0.253', u'platform': u'x86_64', u'host': u'comp-a09',
u'do_local_attach': False, u'mountpoint': u'/dev/vda', u'os_type':
u'linux2', u'multipath': True}.
2021-04-23 16:34:58.111 17298 INFO
cinder.volume.drivers.huawei.huawei_driver [req-77231901-21bf-
42c3-99a3-5a2e0b13a066 7d52a2a770e4402982390eb658425083
ab5b748ab5e04fa58c8b4aad9f744e2b - default default] Terminate iscsi
connection for volume de2c38b7-bf67-42b7-ae51-76fca6c7c05b, connector
info {u'initiator': u'iqn.1993-08.org.debian:01:36b34c8cbdd8', u'ip':
u'10.10.0.148', u'platform': u'x86_64', u'host': u'comp-a03',
u'do_local_attach': False, u'mountpoint': u'/dev/vda', u'os_type':
u'linux2', u'multipath': True}.
In case some additional logs are needed, just let me know.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1925818/+subscriptions
References
