yahoo-eng-team team mailing list archive

[Launchpad Bug Tracker <1725067@xxxxxxxxxxxxxxxxxx>]

This bug was fixed in the package cloud-init -
17.1-27-geb292c18-0ubuntu1~16.04.1

---------------
cloud-init (17.1-27-geb292c18-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * New upstream snapshot.
    - EC2: Limit network config to fallback nic, fix local-ipv4 only
      instances. (LP: #1728152)
    - Gentoo: Use "rc-service" rather than "service". [Carlos Konstanski]

cloud-init (17.1-25-g17a15f9e-0ubuntu1~16.04.1) xenial-proposed;
urgency=medium

  * New upstream snapshot.
    - resizefs: Fix regression when system booted with root=PARTUUID=
      (LP: #1725067)
    - tools: make yum package installation more reliable
    - citest: fix remaining warnings raised by integration tests.
    - citest: show the class actual class name in results.
    - ntp: fix config module schema to allow empty ntp config
      (LP: #1724951)
    - tools: disable fastestmirror if using proxy [Joshua Powers]

cloud-init (17.1-18-gd4f70470-0ubuntu1~16.04.2) xenial-proposed;
urgency=medium

  * cherry-pick 41152f1: schema: Log debug instead of warning when
    jsonschema is absent (LP: #1724354)

cloud-init (17.1-18-gd4f70470-0ubuntu1~16.04.1) xenial-proposed;
urgency=medium

  * drop the following cherry picks, now incorporated in snapshot.
    + debian/patches/cpick-a2f8ce9c-Do-not-provide-systemd-fsck-drop...
  * debian/copyright: dep5 updates, reorganize, add Apache 2.0 license.
    (LP: #1718681)
  * debian/control: drop dependency on python3-prettytable
  * debian/rules: install rsyslog file with 0644 mode instead of 0755.
  * debian/rules, debian/apport-launcher.py: add an apport hook. (LP: #1607345)
  * New upstream snapshot. (LP: #1721847)
    - simpletable: Fix get_string method to return table-formatted string
    - net: Handle bridge stp values of 0 and convert to boolean type
      [Chad Smith]
    - tools: Give specific --abbrev=8 to "git describe"
    - network: bridge_stp value not always correct [Ryan Harper]
    - tests: re-enable tox with nocloud-kvm support [Joshua Powers]
    - systemd: remove limit on tasks created by cloud-init-final.service.
      [Robert Schweikert]
    - suse: Support addition of zypper repos via cloud-config.
      [Robert Schweikert]
    - tests: Combine integration configs and testcases [Joshua Powers]
    - Azure, CloudStack: Support reading dhcp options from systemd-networkd.
      [Dimitri John Ledkov]
    - packages/debian/copyright: remove mention of boto and MIT license
    - systemd: only mention Before=apt-daily.service on debian based distros.
      [Robert Schweikert]
    - Add missing simpletable and simpletable tests for failed merge
      [Chad Smith]
    - Remove prettytable dependency, introduce simpletable [Andrew Jorgensen]
    - debian/copyright: dep5 updates, reorganize, add Apache 2.0 license.
      [Joshua Powers]
    - tests: remove dependency on shlex [Joshua Powers]
    - AltCloud: Trust PATH for udevadm and modprobe.
    - DataSourceOVF: use util.find_devs_with(TYPE=iso9660)
      [Ryan Harper]
    - tests: remove a temp file used in bootcmd tests.
    - release 17.1
    - doc: document GCE datasource. [Arnd Hannemann]
    - suse: updates to templates to support openSUSE and SLES.
      [Robert Schweikert]
    - suse: Copy sysvinit files from redhat with slight changes.
      [Robert Schweikert]
    - docs: fix sphinx module schema documentation [Chad Smith]
    - tests: Add cloudinit package to all test targets [Chad Smith]
    - Makefile: No longer look for yaml files in obsolete ./bin/.
    - tests: fix ds-identify unit tests to set EC2_STRICT_ID_DEFAULT.
    - ec2: Fix maybe_perform_dhcp_discovery to use /var/tmp as a tmpdir
      [Chad Smith]
    - Azure: wait longer for SSH pub keys to arrive.
      [Paul Meyer]
    - GCE: Fix usage of user-data.
    - cmdline: add collect-logs subcommand. [Chad Smith]
    - CloudStack: consider dhclient lease files named with a hyphen.
    - resizefs: Drop check for read-only device file, do not warn on
      overlayroot. [Chad Smith]
    - tests: Enable the NoCloud KVM platform [Joshua Powers]
    - resizefs: pass mount point to xfs_growfs [Dusty Mabe]
    - vmware: Enable nics before sending the SUCCESS event. [Sankar Tanguturi]
    - cloud-config modules: honor distros definitions in each module
      [Chad Smith]
    - chef: Add option to pin chef omnibus install version
      [Ethan Apodaca]
    - tests: execute: support command as string [Joshua Powers]
    - schema and docs: Add jsonschema to resizefs and bootcmd modules
      [Chad Smith]
    - tools: Add xkvm script, wrapper around qemu-system [Joshua Powers]
    - vmware customization: return network config format
      [Sankar Tanguturi]
    - Ec2: only attempt to operate at local mode on known platforms.
    - Use /run/cloud-init for tempfile operations.
    - ds-identify: Make OpenStack return maybe on arch other than intel.
    - tests: mock missed openstack metadata uri network_data.json
      [Chad Smith]
    - relocate tests/unittests/helpers.py to cloudinit/tests
      [Lars Kellogg-Stedman]
    - tox: add nose timer output [Joshua Powers]
    - upstart: do not package upstart jobs, drop ubuntu-init-switch module.
    - tests: Stop leaking calls through unmocked metadata addresses
      [Chad Smith]
    - distro: allow distro to specify a default locale [Ryan Harper]
    - tests: fix two recently added tests for sles distro.
    - url_helper: dynamically import oauthlib import from inside oauth_headers
      [Chad Smith]
    - tox: make xenial environment run with python3.6
    - suse: Add support for openSUSE and return SLES to a working state.
      [Robert Schweikert]
    - GCE: Add a main to the GCE Datasource.
    - ec2: Add IPv6 dhcp support to Ec2DataSource. [Chad Smith]
    - url_helper: fail gracefully if oauthlib is not available
      [Lars Kellogg-Stedman]
    - cloud-init analyze: fix issues running under python 2. [Andrew Jorgensen]
    - Configure logging module to always use UTC time.
      [Ryan Harper]
    - Log a helpful message if a user script does not include shebang.
      [Andrew Jorgensen]
    - cli: Fix command line parsing of coniditionally loaded subcommands.
      [Chad Smith]
    - doc: Explain error behavior in user data include file format.
      [Jason Butz]
    - cc_landscape & cc_puppet: Fix six.StringIO use in writing configs
      [Chad Smith]
    - schema cli: Add schema subcommand to cloud-init cli and cc_runcmd schema
      [Chad Smith]
    - Debian: Remove non-free repositories from apt sources template.
      [Joonas Kylmälä]
    - tools: Add tooling for basic cloud-init performance analysis.
      [Chad Smith]
    - network: add v2 passthrough and fix parsing v2 config with bonds/bridge
      params [Ryan Harper]
    - doc: update capabilities with features available, link doc reference,
      cli example [Ryan Harper]
    - vcloud directory: Guest Customization support for passwords
      [Maitreyee Saikia]
    - ec2: Allow Ec2 to run in init-local using dhclient in a sandbox.
      [Chad Smith]
    - cc_ntp: fallback on timesyncd configuration if ntp is not installable
      [Ryan Harper]
    - net: Reduce duplicate code. Have get_interfaces_by_mac use
      get_interfaces.
    - tests: Fix build tree integration tests [Joshua Powers]
    - sysconfig: Dont repeat header when rendering resolv.conf
      [Ryan Harper]
    - archlinux: Fix bug with empty dns, do not render 'lo' devices.

 -- Chad Smith <chad.smith@xxxxxxxxxxxxx>  Tue, 31 Oct 2017 13:10:51
-0600

** Changed in: cloud-init (Ubuntu Xenial)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1725067

Title:
  cloud-init resizefs fails when booting with root=PARTUUID=

Status in cloud-init:
  Fix Released
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Xenial:
  Fix Released
Status in cloud-init source package in Zesty:
  Fix Released
Status in cloud-init source package in Artful:
  Fix Released

Bug description:
  http://pad.lv/1725067
  https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1725067

  === Begin SRU Template ===
  [Impact]
  Growing the root partition would not resize and leave a Traceback in /var/log/cloud-init.log if both:
   a.) The device /dev/root did not exist
       -- this case only hit xenial only because zesty already had a /dev/root created in the image so resize succeeded.
   b.) the kernel command line included PARTUUID=<value>
       -- This potential SRU regression occurred because of code restructuring that still discovered (and logged) but didn't use the discovered kernel_cmdline root device path when resizing.

  [Test Case]
  get-proposed-image is
    https://github.com/cloud-init/qa-scripts/blob/master/scripts/get-proposed-cloudimg
  It downloads a cloud image of a given release, and then creates a -proposed
  image with cloud-init upgraded.

  A script 'recreate.sh' will run each of the steps below automated.
    https://github.com/cloud-init/ubuntu-sru/blob/master/bugs/lp-1684869/recreate.sh

  NOTE: By default the images downloaded start off as 2Gig partitions,
  by requesting a 10G sparse qcow2 image resizefs should find the
  partition and resize it to the full 10Gig. If resizefs fails, we'd
  still be stuck with a 2GB response for "df -h /". This is what failed
  on our previous SRU proposal. The disk stayed at 2G instead of being
  resized to 10G.

  1.) get a (proposed) disk image image.
    and convert it to raw so you can read the partuuid with sfdisk
    (get-proposed-image does this, if not,
    'qemu-img convert -O raw orig.img orig.raw')
    ./get-proposed-image

  2.) get the partition uuid of the first partition
     # for xenial images that are dos partition table rather than gpt
     # we need to convert that with:
     #    sgdisk --mbrtogpt $raw
     $ raw=yakkety-server-cloudimg-amd64-proposed.raw
     $ ptuuid=$(sfdisk --part-uuid $raw 1)

  3.) create a nocloud seed
     $ printf "%s\n%s\n%s\n%s\n" "#cloud-config" "password: passw0rd" \
          "chpasswd: {expire: False}" "ssh_pwauth: True" > my-user-data
     $ echo "instance-id: $(uuidgen || echo i-abcdefg)" > my-meta-data
     $ cloud-localds my-seed.img my-user-data my-meta-data

  4.) extract kernel from inside the image
     $ sudo mount-image-callback $raw -- mchroot sh -xc 'cat /boot/vmlinu?-*' > kernel

  5.) boot instance with disk backed by the raw disk above.

     $ qemu-img create -f qcow2 -b $raw disk.img 10G
     $ qemu-system-x86_64 -enable-kvm \
         -drive file=disk.img,if=ide,index=0 -drive file=my-seed.img,if=ide \
         -net nic -net user,hostfwd=tcp::2222-:22 \
         -snapshot -m 768 -nographic -echr 0x05 \
         -kernel kernel \
         -append "root=PARTUUID=${ptuuid} ro console=tty1 console=ttyS0"

  6.) log in, verify / has been resized.
     log in with 'ubuntu' and password 'passw0rd'
      $ df -h /
      Filesystem      Size  Used Avail Use% Mounted on
      /dev/root       9.6G 1009M  8.6G  11% /

  [Regression Potential]
  Regressions would surface as the root filesystem not being correctly resized.
  The user would find themselves with not as much disk as expected.

  [Other Info]
  The qemu-system-x86 command above uses ide devices.  This is because
  the ide device emulated by qemu is built into the -generic kernel,
  while the more common virtio-block or virtio-scsi are not.  If you
  attach those device types, it will fail with 'cant find root'.

  Note that this was a regression of changes added in
   * bug 1684869: growing root partition does not always work with root=PARTUUID=
   * bug 1677376: growing partitions does not work when booted without initramfs

  The issue probably is only seen if using the version of cloud-init
  in xenial-proposed.  Zesty and artful kernels or userspace made the change
  actually not regress.  However we will verify functionality for the uploaded
  version in each of x, z, a.

  [Other Info]
  Upstream commit at
    https://git.launchpad.net/cloud-init/commit/?id=17a15f9e0a

  === End SRU Template ===

  A freshly built Ubuntu 17.10 image that's configured to boot without
  initramfs by passing root=PARTUUID=<uuid> on the kernel commandline
  comes up degraded because cloud-init fails to resize the root
  partition.

  Oct 20 00:07:30 ubuntu cloud-init[493]: 2017-10-20 00:07:30,656 - util.py[WARNING]: Failed to resize filesystem (cmd=('resize2fs', '/dev/root'))
  Oct 20 00:07:30 ubuntu cloud-init[493]: 2017-10-20 00:07:30,662 - util.py[WARNING]: Running module resizefs (<module 'cloudinit.config.cc_resizefs' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_resizefs.py'>) failed

  Looking at the code, I see that there are two separate implementations
  of rootdev_from_cmdline, one in cloudinit/util.py and one in
  cloudinit/config/cc_resizefs.py; and the second does not handle
  partuuid.

  >>> from cloudinit.util import rootdev_from_cmdline
  >>> rootdev_from_cmdline('BOOT_IMAGE=/boot/vmlinuz-4.4.0-1007-kvm root=PARTUUID=f122607f-3631-4411-bd34-de4bed76e0f7 ro console=tty1 console=ttyS0 systemd.log_level=debug')
  '/dev/disk/by-partuuid/f122607f-3631-4411-bd34-de4bed76e0f7'
  >>> from cloudinit.config.cc_resizefs import rootdev_from_cmdline
  >>> rootdev_from_cmdline('OT_IMAGE=/boot/vmlinuz-4.4.0-1007-kvm root=PARTUUID=f122607f-3631-4411-bd34-de4bed76e0f7 ro console=tty1 console=ttyS0 systemd.log_level=debug')
  '/dev/PARTUUID=f122607f-3631-4411-bd34-de4bed76e0f7'
  >>>

  This is related to bug #1684869; I'm not sure if it's the same bug
  reintroduced or if was never fixed properly on trunk (17.1).

  Related bugs:
   * bug 1684869: growing root partition does not always work with root=PARTUUID=
   * bug 1685291: RFC: virtio and virtio-scsi should be built in
   * bug 1677376: growing partitions does not work when booted without initramfs

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1725067/+subscriptions