yahoo-eng-team team mailing list archive

[Launchpad Bug Tracker <1687712@xxxxxxxxxxxxxxxxxx>]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-
0ubuntu1~16.10.1

---------------
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~16.10.1) yakkety-proposed; urgency=medium

  * New upstream snapshot.
    - net: fix reading and rendering addresses in cidr format.
      [Dimitri John Ledkov] (LP: #1689346, #1684349)
    - disk_setup: udev settle before attempting partitioning or fs creation.
      (LP: #1692093)
    - GCE: Update the attribute used to find instance SSH keys.
      [Daniel Watkins] (LP: #1693582)
    - nplan: For bonds, allow dashed or underscore names of keys.
      [Dimitri John Ledkov] (LP: #1690480)
    - tests: python2.6: fix unit tests usage of assertNone and format.
    - tests: update docstring on test_configured_list_with_none
    - fix tools/ds-identify to not write None twice.
    - tox/build: do not package depend on style requirements.
    - tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration.  [Andreas Karis]
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - tests: Fix unittest bug in ntp tests. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - tests: fix cloudstack unit tests to avoid accessing
      /var/lib/NetworkManager [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present.
      [Chris Brinker] (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - make deb: Add devscripts dependency for make deb.
      Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network configuration defaults policy and formats.
      [Ryan Harper]
    - doc: Fix name of "uri" key in docs for "cc_apt_configure" module
      [Felix Dreissig]
    - tests: Enable artful in integration tests [Joshua Powers]

 -- Scott Moser <smoser@xxxxxxxxxx>  Fri, 26 May 2017 16:08:21 -0400

** Changed in: cloud-init (Ubuntu Yakkety)
       Status: Fix Committed => Fix Released

** Changed in: cloud-init (Ubuntu Xenial)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1687712

Title:
  cc_disk_setup: fs_setup with cmd doesn't work

Status in cloud-init:
  Confirmed
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Xenial:
  Fix Released
Status in cloud-init source package in Yakkety:
  Fix Released
Status in cloud-init source package in Zesty:
  Fix Released

Bug description:
  === Begin SRU Template ===
  [Impact]
  If the user specifies cloud-config with a 'fs_setup' entry containing
  a 'cmd', then warning will appear in cloud-init.log and expected filesystem
  will not be created.

  This is because cloud-init would essentially try to execute a name like:
    "mkfs -t TYPE -L LABEL DEVICE"
  rather than a name 'mkfs' with arguments '-t', TYPE, ...
  No split was done on space.  The fix was to enable
  shell intrepretation so that the split would be done.

  [Test Case]
  This test case assumes a disk will be attached named '/dev/vdb'.
  You can change the 'dev=' to be 'sdb' if that will be the device name.
  The test cases boots an instance, ads proposed and then 'cleans' the
  instance, but similarly this will work if the config is provided
  in user-data.

  $ dev=vdb
  $ sudo tee /etc/cloud/cloud.cfg.d/disk-setup.cfg <<EOF
  #cloud-config
  disk_setup:
    /dev/$dev:
      table_type: gpt
      layout: [[100, 83]]
   
  fs_setup:
   - cmd: mkfs -F -t %(filesystem)s -L %(label)s %(device)s
     filesystem: ext4
     device: /dev/$dev
     partition: 1
     label: repro

  mounts:
   - [/dev/${dev}1, /mnt]
  EOF

  $ sudo rm -Rf /var/lib/cloud /var/log/cloud-init*
  ## remove the old entry in /etc/fstab, which can cause LP: #1691489
  $ sudo sed -i.dist '/comment=cloudconfig/d' /etc/fstab

  ## wipe the disk so that we make sure we format it.
  $ sudo python3 -c "import sys;
  buf = b'\\0' * 1024 * 1024 * 8
  with open(sys.argv[1], 'wb+') as fp:
     fp.write(buf)
     fp.seek(-(len(buf)), 2)
     fp.write(buf)" /dev/$dev

  $ sudo reboot

  
  ## Now ssh back in, and expect to have a filesystem on /dev/vdb1
  ## that is mounted at /mnt

  [Regression Potential]
  Regression could occur if a user had provided a string to the 'cmd'
  argument that had special shell characters in it.
  For example:
   cmd: "/my/cmd *foo*"

  That would previously have executed the command "/mnt/cmd *foo*", but
  will now execute the command /mnt/cmd with argument shell filename
  expansion of *foo*

  [Other Info]
  Upstream commit at
    https://git.launchpad.net/cloud-init/commit/?id=4f0f171c2

  === End SRU Template ===


  This reproduces on Azure, but it should fail similarly elsewhere.
  Consider repro.yml:

  #cloud-config
  fs_setup:
      - special:
        cmd: mkfs -t %(filesystem)s -L %(label)s %(device)s
        filesystem: ext4
        device: /dev/sdb1
        label: repro

  Create a VM with this cloud config:
  $ az vm create -g $rg -l westus2 --custom-data @repro.yml --image UbuntuLTS -n repro2

  Then cloud-init will fail with:
  Failed to exec of 'mkfs -t ext4 -L repro /dev/sdb1':
  Unexpected error while running command.
  Command: mkfs -t ext4 -L repro /dev/sdb1
  Exit code: -
  Reason: [Errno 2] No such file or directory: 'mkfs -t ext4 -L repro /dev/sdb1'

  $ dpkg-query -W -f='${Version}' cloud-init
  0.7.9-48-g1c795b9-0ubuntu1~16.04.1

  Bug is in mkfs() in cc_disk_setup.py, which creates a shell-like
  string in the case that cmd was specified and a exec-like array in the
  other case (around line 913).

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1687712/+subscriptions