yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #70512
[Bug 1739593] Re: Swapping encrypted volumes can lead to data loss and a possible compute host DOS attack
Reviewed: https://review.openstack.org/460243
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=cd3eb60c2c00bcccfa9ccd4bf9d1a96ae7a5cd88
Submitter: Zuul
Branch: master
commit cd3eb60c2c00bcccfa9ccd4bf9d1a96ae7a5cd88
Author: Lee Yarwood <lyarwood@xxxxxxxxxx>
Date: Mon Apr 24 19:23:44 2017 +0100
libvirt: Collocate encryptor and volume driver calls
This change introduces new utility methods for attaching and detaching
frontend volume encryptors. These methods centralise the optional
fetching of encryption metadata associated with a volume, fetching of the
required encryptor and calls to detach or attach the encryptor.
These new utility methods are called either after initially connecting
to or before disconnecting from a volume. This ensures encryptors are
correctly connected when swapping volumes for example, where previously
no attempt was made to attach an encryptor to the target volume.
The request context is provided to swap_volume and various other config
generation related methods to allow for the lookup of the relevant
encryption metadata if it is not provided.
Closes-bug: #1739593
Change-Id: Ica323b87fa85a454fca9d46ada3677f18fe50022
** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1739593
Title:
Swapping encrypted volumes can lead to data loss and a possible
compute host DOS attack
Status in OpenStack Compute (nova):
Fix Released
Status in OpenStack Security Advisory:
Incomplete
Bug description:
Description
===========
At present when swapping encrypted volumes no attempt is made to
attach an encryptor to the target volume. This results in the RAW
underlying volume being used during the rebase, where decrypted data
is copied from the original volume to the target:
https://github.com/openstack/nova/blob/76dfdfc1ad8c0e5376bd997e45f65bec9ff53d12/nova/virt/libvirt/driver.py#L1338-L1372
Any attempt to detach and then reattach this volume from the instance
will lead to the volume being reformatted as the os-brick supplied
encryptors do not identify the volume as encrypted:
https://github.com/openstack/os-
brick/blob/6835b885dc4144fdc6e9863ca59ae54f76938995/os_brick/encryptors/luks.py#L138-L161
Additionally, while unlikely, a malicious user could easily DOS the
compute node hosting the instance by writing a corrupt LUKS header to
the RAW volume before detaching and reattaching the volume. For
example, setting a keyslot iters (used by PBKDF2) to a large value etc
(kudos to mdbooth for suggesting this):
https://gitlab.com/cryptsetup/cryptsetup/wikis/LUKS-standard/on-disk-
format.pdf
This method of DOS'ing the compute host was previously discussed in
the context of bug 1724573 but dismissed as access to the underlying
volume was dependent on a host reboot, outside of a users control.
This bug differs as a user has full control of the above volume-
update/swap_volume flow that provides access to the underlying volume.
Steps to reproduce
==================
- Create two encrypted volumes
$ cinder type-create LUKS
$ cinder encryption-type-create --cipher aes-xts-plain64 \
--key_size 256 \
--control_location front-end LUKS luks
$ cinder type-create LUKS_NEW
$ cinder encryption-type-create --cipher aes-xts-plain64 \
--key_size 256 \
--control_location front-end LUKS_NEW luks
$ cinder create --volume-type LUKS 1
$ cinder create --volume-type LUKS_NEW 1
- Spawn an instance, attaching the first volume before swapping to the
second:
$ nova boot --image cirros-0.3.5-x86_64-disk --flavor 1 swap_test
$ nova volume-attach $instance $vol-luks
$ nova volume-update $instance $vol-luks $vol-luks-new
- Review the resulting volume attachment on the compute host:
$ virsh domblklist $instance
Target Source
------------------------------------------------
vda /opt/stack/data/nova/instances/3d4c5842-45ab-4660-bf6e-9459f9a2ff8a/disk
vdb /dev/disk/by-id/scsi-36001405ba072cc9f93e444c9433ead1c
$ ll /dev/disk/by-id/scsi-36001405ba072cc9f93e444c9433ead1c
lrwxrwxrwx. 1 root root 9 Dec 21 05:30 /dev/disk/by-id/scsi-36001405ba072cc9f93e444c9433ead1c -> ../../sdd
$ sudo qemu-img info /dev/disk/by-id/scsi-36001405ba072cc9f93e444c9433ead1c
image: /dev/disk/by-id/scsi-36001405ba072cc9f93e444c9433ead1c
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: 0
Expected result
===============
The encrypted volumes are rebased with their associated encryptors
attached, leading to encrypted data being written to the underlying
volumes.
Actual result
=============
Decrypted data from the source volume is written to the underlying
target volume. This data will be lost with a subsequent detach /
attach cycle. Access to the underlying volume could also be used by a
malicious user to DOS the local compute host.
Environment
===========
1. Exact version of OpenStack you are running. See the following
list for all releases: http://docs.openstack.org/releases/
2. Which hypervisor did you use?
(For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
What's the version of that?
Libvirt + KVM
2. Which storage type did you use?
(For example: Ceph, LVM, GPFS, ...)
What's the version of that?
LVM
3. Which networking type did you use?
(For example: nova-network, Neutron with OpenVSwitch, ...)
N/A
Logs & Configs
==============
N/A
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1739593/+subscriptions