yahoo-eng-team team mailing list archive

[James Falcon <1841466@xxxxxxxxxxxxxxxxxx>]

This bug is believed to be fixed in cloud-init in version 20.3. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
       Status: Triaged => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1841466

Title:
  ds-identify fails to detect NoCloud datastore with LABEL_FATBOOT
  instead of LABEL (change introduced recently in util-linux-2.33-rc1)

Status in cloud-init:
  Fix Released

Bug description:
  Original bug report with detailed description was created for Xen
  Orchestra here: https://github.com/vatesfr/xen-orchestra/issues/4449

  
  Brief description:

  On systems with util-linux-2.33-rc1 or younger (e.g. Debian 10
  Buster), ds-identify fails to detect when a disk of a NoCloud
  datasource has a label written to the boot sector of the disk. Before
  util-linux-2.33-rc1, blkid showed "LABEL=cidata". With the change,
  blkid shows "LABEL_FATBOOT=cidata" (newly introduced / additional
  label).

  I ran into this when using cloud-init together with Xen Orchestra
  v5.48 (Xen Orchestra is a management interface for xen; in my case
  xcp-ng v8.0.0). I created a VM template based on the recently released
  Debian 10.0 Buster, which uses util-linux 2.33.1. Upon boot, ds-
  identify fails to detect the NoCloud datasource / virtual disk which
  Xen Orchestra generated (the disk is created with this code from
  https://github.com/natevw/fatfs. With an older Debian 8 (util-
  linux-2.25.0) based template, ds-identify detects the NoCloud
  datasource disk fine.

  
  Longer description / likely cause of the problem:

  Xen Orchestra creates the NoCloud as a partition-less disk with a
  FAT16 filesystem which has the NoCloud user-data and meta-data files.
  The label "cidata" is written into the boot sector of the virtual
  disk. With the same disk, older versions of blkid report
  "LABEL=cidata" whereas newer versions detect "LABEL_FATBOOT=cidata".
  The ds-identify shell script checks only for the presence of the field
  called "LABEL" and not for "LABEL_FATBOOT".

  Relevant commit message from the util-linux-2.33-rc1 changelog (commit
  f0ca7e80d7a171701d0d04a3eae22d97f15d0683):
  https://cdn.kernel.org/pub/linux/utils/util-
  linux/v2.33/v2.33-rc1-ChangeLog

  libblkid: vfat: Change parsing label in special cases

  * Use only label from the root directory and do not fallback to the
  label stored in boot sector. This is how MS-DOS 6.22, MS-DOS 7.10,
  Windows 98, Windows XP and also Windows 10 behave. Moreover Windows XP
  and Windows 10 do not touch label in boot sector anymore, so removing
  FAT label on those Windowses leads to having old label still stored in
  boot sector (which MS-DOS and Windows fully ignore).

  * Label entry "NO NAME" in root directory is treated as label "NO
  NAME" instead of empty label. In root directory it has no special
  meaning. String "NO NAME" has a special meaning (empty label) only for
  label stored in boot sector.

   * Label from the boot sector is now stored into LABEL_FATBOOT field.
  So if there are applications which depends or needs to read this
  label, they have ability.

  * After this change LABEL always correspondent to the label from the
  root directory and LABEL_FATBOOT to the label stored in the boot
  sector. If some of those labels is missing or is not present (e.g. "NO
  LABEL" in boot sector) then particular field is not set.

  Possible fix:

  I did a trivial change of 2 lines to ds-identify to check for
  LABEL_FATBOOT after the check for LABEL:

  debian@cloudbuster:~$ diff /usr/lib/cloud-init/ds-identify.ORIGINAL /usr/lib/cloud-init/ds-identify
  236a237,238
  > LABEL_FATBOOT=*) label="${line#LABEL_FATBOOT=}";
  >                  labels="${labels}${line#LABEL_FATBOOT=}${delim}";;

  For me this solves the problem, as in: the cloud-init enabled VM boots
  up, ds-identify finds "LABEL_FATBOOT=cidata" and cloud-init correctly
  executes. In cases where both labels are written, the latter over-
  writes the former, which could be a theoretical problem if the values
  differ, but I am not sure how likely this case is.

  Further debug information as requested by @rharper on IRC:

  - cloud-init.tar.gz (Debian 10 / ds-identify fail)

  - Debian version:

  debian@cloudbuster:~$ lsb_release -a
  No LSB modules are available.
  Distributor ID:	Debian
  Description:	Debian GNU/Linux 10 (buster)
  Release:	10
  Codename:	buster

  - util-linux version:

  debian@cloudbuster:~$ sudo blkid -V
  blkid from util-linux 2.33.1  (libblkid 2.33.1, 09-Jan-2019)

  - blkid output:

  debian@cloudbuster:~$ sudo blkid /dev/xvdb
  /dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT="cidata" UUID="355A-4FC2" TYPE="vfat"

  - udevadm outout:
  debian@cloudbuster:~$ udevadm info --query=all /sys/class/block/xvdb
  P: /devices/vbd-832/block/xvdb
  N: xvdb
  L: 0
  S: disk/by-uuid/355A-4FC2
  E: DEVPATH=/devices/vbd-832/block/xvdb
  E: DEVNAME=/dev/xvdb
  E: DEVTYPE=disk
  E: MAJOR=202
  E: MINOR=16
  E: SUBSYSTEM=block
  E: USEC_INITIALIZED=4239917
  E: ID_FS_UUID=355A-4FC2
  E: ID_FS_UUID_ENC=355A-4FC2
  E: ID_FS_VERSION=FAT16
  E: ID_FS_TYPE=vfat
  E: ID_FS_USAGE=filesystem
  E: DEVLINKS=/dev/disk/by-uuid/355A-4FC2
  E: TAGS=:systemd:

  # Some experiments:

  - This is interesting - dosfslabel incorrectly reports the label,
  while blkid (above) clearly shows the field is empty / not set:

  debian@cloudbuster:~$ sudo dosfslabel /dev/xvdb
  cidata

  - Here I am first setting the label with dosfslabel to see what
  happens and then check blkid again:

  debian@cloudbuster:~$ sudo dosfslabel /dev/xvdb cidata
  fatlabel: warning - lowercase labels might not work properly with DOS or Windows

  debian@cloudbuster:~$ sudo blkid /dev/xvdb
  /dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT="cidata" LABEL="cidata" UUID="355A-4FC2" TYPE="vfat"
  # Now blkid reports both labels

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1841466/+subscriptions