sts-sponsors team mailing list archive

[Launchpad Bug Tracker <1879980@xxxxxxxxxxxxxxxxxx>]

You have been subscribed to a public bug by Guilherme G. Piccoli (gpiccoli):

[Impact]
* Considering a setup of a encrypted rootfs on top of md RAID1 device, Ubuntu is currently unable to decrypt the rootfs if the array gets degraded, like for example if one of the array's members gets removed.

* The problem has 2 main aspects: first, cryptsetup initramfs script
attempts to decrypt the array only in the local-top boot stage, and in
case it fails, it gives-up and show user a shell (boot is aborted).

* Second, mdadm initramfs script that assembles degraded arrays executes
later on boot, in the local-block stage. So, in a stacked setup of
encrypted root on top of RAID, if the RAID is degraded, cryptsetup fails
early in the boot, preventing mdadm to assemble the degraded array.

* The hereby proposed solution has 2 components: first, cryptsetup
script is modified to allow a gentle failure on local-top stage, then it
retries for a while (according to a heuristic based on ROOTDELAY with
minimum of 30 executions) in a later stage (local-block). This gives
time to other initramfs scripts to run, like mdadm in local-block stage.
And this is meant to work this way according to initramfs-tools
documentation (although Ubuntu changed it a bit with wait-for-root,
hence we stopped looping on local-block, see next bullet).

* Second, initramfs-tools was adjusted - currently, it runs for a while
the mdadm local-block script, in order to assemble the arrays in a non-
degraded mode. We extended this approach to also execute cryptsetup, in
a way that after mdadm ends its execution, we execute at least once more
time cryptsetup. In an ideal world we should loop on local-block as
Debian's initramfs (in a way to remove hardcoded mdadm/cryptsetup
mentions from initramfs-tools code), but this would be really a big
change, non-SRUable probably. I plan to work that for future Ubuntu
releases.

[Test case]
* Install Ubuntu in a Virtual Machine with 2 disks. Use the installer to create a RAID1 volume and an encrypted root on top of it.

* Boot the VM, and use "sgdisk"/"wipefs" to erase the partition table
from one of the RAID members. Reboot and it will fail to mount rootfs
and continue boot process.

* If using the initramfs-toos/cryptsetup patches hereby proposed, the
rootfs can be mounted normally.

[Regression potential]

* There are potential for regressions, since this is a change in 2 boot
components. The patches were designed in a way to keep the regular case
working, it changes the failure case which is not currently working
anyway.

* A modification in the behavior of cryptsetup was introduced: right
now, if we fail the password 3 times (the default maximum attempts), the
script doesn't "panic" and drop to a shell immediately; instead it runs
once more (or twice, if mdadm is installed) before failing. This is a
minor change given the benefit of the being able to mount rootfs in a
degraded RAID1 scenario.

* Other potential regressions could show-up as boot problems, but the
change in initramfs-tools specifically is not invasive, it just may
delay boot time a bit, given we now run cryptsetup multiple times on
local-block, with 1 sec delays between executions.

** Affects: cryptsetup (Ubuntu)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: In Progress

** Affects: initramfs-tools (Ubuntu)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: In Progress

** Affects: mdadm (Ubuntu)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: Opinion

** Affects: cryptsetup (Ubuntu Xenial)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: Won't Fix

** Affects: initramfs-tools (Ubuntu Xenial)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: Won't Fix

** Affects: mdadm (Ubuntu Xenial)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: Won't Fix

** Affects: cryptsetup (Ubuntu Bionic)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: In Progress

** Affects: initramfs-tools (Ubuntu Bionic)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: In Progress

** Affects: mdadm (Ubuntu Bionic)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: Opinion

** Affects: cryptsetup (Ubuntu Focal)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: In Progress

** Affects: initramfs-tools (Ubuntu Focal)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: In Progress

** Affects: mdadm (Ubuntu Focal)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: Opinion

** Affects: cryptsetup (Ubuntu Groovy)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: In Progress

** Affects: initramfs-tools (Ubuntu Groovy)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: In Progress

** Affects: mdadm (Ubuntu Groovy)
     Importance: Medium
     Assignee: Guilherme G. Piccoli (gpiccoli)
         Status: Opinion

** Affects: cryptsetup (Debian)
     Importance: Unknown
         Status: New


** Tags: sts
-- 
Fail to boot with LUKS on top of RAID1 if the array is broken/degraded
https://bugs.launchpad.net/bugs/1879980
You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report.