yahoo-eng-team team mailing list archive

[Launchpad Bug Tracker <1725067@xxxxxxxxxxxxxxxxxx>]

This bug was fixed in the package cloud-init - 17.1-25-g17a15f9e-
0ubuntu1~17.10.1

---------------
cloud-init (17.1-25-g17a15f9e-0ubuntu1~17.10.1) artful-proposed; urgency=medium

  * New upstream snapshot.
    - resizefs: Fix regression when system booted with root=PARTUUID=
      (LP: #1725067)
    - tools: make yum package installation more reliable
    - citest: fix remaining warnings raised by integration tests.
    - citest: show the class actual class name in results.
    - ntp: fix config module schema to allow empty ntp config
      (LP: #1724951)
    - tools: disable fastestmirror if using proxy [Joshua Powers]
    - schema: Log debug instead of warning when jsonschema is not available.
      (LP: #1724354)

 -- Chad Smith <chad.smith@xxxxxxxxxxxxx>  Mon, 23 Oct 2017 15:07:35
-0600

** Changed in: cloud-init (Ubuntu Artful)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1725067

Title:
  cloud-init resizefs fails when booting with root=PARTUUID=

Status in cloud-init:
  Fix Released
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Xenial:
  Fix Committed
Status in cloud-init source package in Zesty:
  Fix Committed
Status in cloud-init source package in Artful:
  Fix Released

Bug description:
  http://pad.lv/1725067
  https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1725067

  === Begin SRU Template ===
  [Impact]
  Growing the root partition would not resize and leave a Traceback in /var/log/cloud-init.log if both:
   a.) The device /dev/root did not exist
       -- this case only hit xenial only because zesty already had a /dev/root created in the image so resize succeeded.
   b.) the kernel command line included PARTUUID=<value>
       -- This potential SRU regression occurred because of code restructuring that still discovered (and logged) but didn't use the discovered kernel_cmdline root device path when resizing.

  [Test Case]
  get-proposed-image is
    https://github.com/cloud-init/qa-scripts/blob/master/scripts/get-proposed-cloudimg
  It downloads a cloud image of a given release, and then creates a -proposed
  image with cloud-init upgraded.

  A script 'recreate.sh' will run each of the steps below automated.
    https://github.com/cloud-init/ubuntu-sru/blob/master/bugs/lp-1684869/recreate.sh

  NOTE: By default the images downloaded start off as 2Gig partitions,
  by requesting a 10G sparse qcow2 image resizefs should find the
  partition and resize it to the full 10Gig. If resizefs fails, we'd
  still be stuck with a 2GB response for "df -h /". This is what failed
  on our previous SRU proposal. The disk stayed at 2G instead of being
  resized to 10G.

  1.) get a (proposed) disk image image.
    and convert it to raw so you can read the partuuid with sfdisk
    (get-proposed-image does this, if not,
    'qemu-img convert -O raw orig.img orig.raw')
    ./get-proposed-image

  2.) get the partition uuid of the first partition
     # for xenial images that are dos partition table rather than gpt
     # we need to convert that with:
     #    sgdisk --mbrtogpt $raw
     $ raw=yakkety-server-cloudimg-amd64-proposed.raw
     $ ptuuid=$(sfdisk --part-uuid $raw 1)

  3.) create a nocloud seed
     $ printf "%s\n%s\n%s\n%s\n" "#cloud-config" "password: passw0rd" \
          "chpasswd: {expire: False}" "ssh_pwauth: True" > my-user-data
     $ echo "instance-id: $(uuidgen || echo i-abcdefg)" > my-meta-data
     $ cloud-localds my-seed.img my-user-data my-meta-data

  4.) extract kernel from inside the image
     $ sudo mount-image-callback $raw -- mchroot sh -xc 'cat /boot/vmlinu?-*' > kernel

  5.) boot instance with disk backed by the raw disk above.

     $ qemu-img create -f qcow2 -b $raw disk.img 10G
     $ qemu-system-x86_64 -enable-kvm \
         -drive file=disk.img,if=ide,index=0 -drive file=my-seed.img,if=ide \
         -net nic -net user,hostfwd=tcp::2222-:22 \
         -snapshot -m 768 -nographic -echr 0x05 \
         -kernel kernel \
         -append "root=PARTUUID=${ptuuid} ro console=tty1 console=ttyS0"

  6.) log in, verify / has been resized.
     log in with 'ubuntu' and password 'passw0rd'
      $ df -h /
      Filesystem      Size  Used Avail Use% Mounted on
      /dev/root       9.6G 1009M  8.6G  11% /

  [Regression Potential]
  Regressions would surface as the root filesystem not being correctly resized.
  The user would find themselves with not as much disk as expected.

  [Other Info]
  The qemu-system-x86 command above uses ide devices.  This is because
  the ide device emulated by qemu is built into the -generic kernel,
  while the more common virtio-block or virtio-scsi are not.  If you
  attach those device types, it will fail with 'cant find root'.

  Note that this was a regression of changes added in
   * bug 1684869: growing root partition does not always work with root=PARTUUID=
   * bug 1677376: growing partitions does not work when booted without initramfs

  The issue probably is only seen if using the version of cloud-init
  in xenial-proposed.  Zesty and artful kernels or userspace made the change
  actually not regress.  However we will verify functionality for the uploaded
  version in each of x, z, a.

  [Other Info]
  Upstream commit at
    https://git.launchpad.net/cloud-init/commit/?id=17a15f9e0a

  === End SRU Template ===

  A freshly built Ubuntu 17.10 image that's configured to boot without
  initramfs by passing root=PARTUUID=<uuid> on the kernel commandline
  comes up degraded because cloud-init fails to resize the root
  partition.

  Oct 20 00:07:30 ubuntu cloud-init[493]: 2017-10-20 00:07:30,656 - util.py[WARNING]: Failed to resize filesystem (cmd=('resize2fs', '/dev/root'))
  Oct 20 00:07:30 ubuntu cloud-init[493]: 2017-10-20 00:07:30,662 - util.py[WARNING]: Running module resizefs (<module 'cloudinit.config.cc_resizefs' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_resizefs.py'>) failed

  Looking at the code, I see that there are two separate implementations
  of rootdev_from_cmdline, one in cloudinit/util.py and one in
  cloudinit/config/cc_resizefs.py; and the second does not handle
  partuuid.

  >>> from cloudinit.util import rootdev_from_cmdline
  >>> rootdev_from_cmdline('BOOT_IMAGE=/boot/vmlinuz-4.4.0-1007-kvm root=PARTUUID=f122607f-3631-4411-bd34-de4bed76e0f7 ro console=tty1 console=ttyS0 systemd.log_level=debug')
  '/dev/disk/by-partuuid/f122607f-3631-4411-bd34-de4bed76e0f7'
  >>> from cloudinit.config.cc_resizefs import rootdev_from_cmdline
  >>> rootdev_from_cmdline('OT_IMAGE=/boot/vmlinuz-4.4.0-1007-kvm root=PARTUUID=f122607f-3631-4411-bd34-de4bed76e0f7 ro console=tty1 console=ttyS0 systemd.log_level=debug')
  '/dev/PARTUUID=f122607f-3631-4411-bd34-de4bed76e0f7'
  >>>

  This is related to bug #1684869; I'm not sure if it's the same bug
  reintroduced or if was never fixed properly on trunk (17.1).

  Related bugs:
   * bug 1684869: growing root partition does not always work with root=PARTUUID=
   * bug 1685291: RFC: virtio and virtio-scsi should be built in
   * bug 1677376: growing partitions does not work when booted without initramfs

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1725067/+subscriptions