curtin-dev team mailing list archive

[Dan Bungert <mp+464246@xxxxxxxxxxxxxxxxxx>]

Dan Bungert has proposed merging ~dbungert/curtin:lp-2057661-flock-zfs-ctd into curtin:master.

Commit message:
do not squash

Requested reviews:
  curtin developers (curtin-dev)
Related bugs:
  Bug #2057661 in subiquity: "install w/ ZFS + encryption: `cryptsetup luksFormat /dev/zvol/rpool/keystore` failure"
  https://bugs.launchpad.net/subiquity/+bug/2057661

For more details, see:
https://code.launchpad.net/~dbungert/curtin/+git/curtin/+merge/464246

zfs: add flock for keystore

In LP: #2057661 and the (currently) 4 duplicates, the luksFormat of
keystore can randomly fail.  stracing of udevd looking for
https://systemd.io/BLOCK_DEVICE_LOCKING/ shows that indeed udevd is
examining these keystore devices.

Lock the keystore, and add some `udevadm settle` to be more like
Ubiquity behavior.

-- 
Your team curtin developers is requested to review the proposed merge of ~dbungert/curtin:lp-2057661-flock-zfs-ctd into curtin:master.

diff --git a/curtin/block/zfs.py b/curtin/block/zfs.py
index ccdddbb..81ba1e1 100644
--- a/curtin/block/zfs.py
+++ b/curtin/block/zfs.py
@@ -12,6 +12,7 @@ from contextlib import ExitStack
 from pathlib import Path
 
 from curtin.config import merge_config
+from curtin.udev import udevadm_settle
 from curtin import distro
 from curtin import util
 from . import blkid, get_supported_filesystems
@@ -89,6 +90,7 @@ class ZPoolEncryption:
         zfs_create(
             self.poolname, "keystore", {"encryption": "off"}, keystore_size,
         )
+        udevadm_settle()
 
         with ExitStack() as es:
             for vdev in self.vdevs:
@@ -97,13 +99,19 @@ class ZPoolEncryption:
             # cryptsetup format and open this keystore
             keystore_volume = f"/dev/zvol/{self.poolname}/keystore"
             cmd = ["cryptsetup", "luksFormat", keystore_volume, self.keyfile]
-            util.subp(cmd)
+
+            # strace has shown that udevd does indeed probe these keystores
+            with util.FlockEx(keystore_volume):
+                util.subp(cmd, capture=True)
+
+            udevadm_settle()
+
             dm_name = f"keystore-{self.poolname}"
             cmd = [
                 "cryptsetup", "open", "--type", "luks", keystore_volume,
                 dm_name, "--key-file", self.keyfile,
             ]
-            util.subp(cmd)
+            util.subp(cmd, capture=True)
 
         with ExitStack() as es:
             # format as ext4, mount it, move the previously-generated systemkey
diff --git a/curtin/util.py b/curtin/util.py
index 66735e8..d9b7962 100644
--- a/curtin/util.py
+++ b/curtin/util.py
@@ -1452,6 +1452,7 @@ class FlockEx:
             raise TimeoutError("Failed to acquire LOCK_EX on {self.device}")
 
     def __exit__(self, *args):
+        LOG.debug(f"Releasing fcntl LOCK_EX on {self.device}")
         with suppress(Exception):
             fcntl.flock(self.lock_fd, fcntl.LOCK_UN)
         with suppress(Exception):