yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1801790] [NEW] Crash running custom command on fs_setup

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: José Riguera <1801790@xxxxxxxxxxxxxxxxxx>
Date: Mon, 05 Nov 2018 20:48:58 -0000
Reply-to: Bug 1801790 <1801790@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Given this cloud-init config:

    fs_setup:
    - cmd:['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sda', '--subvol', 'data:/data']
      label: volumedata
      filesystem: 'btrfs'
      device: '/dev/sda'

Error log:

    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Writing to /var/lib/cloud/instances/raspberrypi/sem/config_disk_setup - wb: [420] 24 bytes
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] helpers.py[DEBUG]: Running config-disk_setup using lock (<FileLock using file '/var/lib/cloud/instances/raspberrypi/sem/config_disk_setup'>)
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: setting up filesystems: [{'label': 'volumedata', 'filesystem': 'btrfs', 'cmd': ['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sda', '--subvol', 'data:/data'], 'device': '/dev/sda'}, {'label': 'volumedata', 'filesystem': 'btrfs', 'cmd': ['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sdb'], 'device': '/dev/sdb'}]
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating new filesystem.
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking /dev/sda against default devices
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using the raw device to place filesystem volumedata on
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: File system type 'btrfs' with label 'volumedata' will be created on /dev/sda
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Creating fs for /dev/sda took 0.003 seconds
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[WARNING]: Failed during filesystem operation
                                           unsupported operand type(s) for %: 'list' and 'dict'
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Failed during filesystem operation
                                           unsupported operand type(s) for %: 'list' and 'dict'
                                           Traceback (most recent call last):
                                             File "/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py", line 155, in handle
                                               func=mkfs, args=(definition,))
                                             File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2181, in log_time
                                               ret = func(*args, **kwargs)
                                             File "/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py", line 917, in mkfs
                                               'device': device,
                                           TypeError: unsupported operand type(s) for %: 'list' and 'dict'
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating new filesystem.
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking /dev/sdb against default devices


Link to the bug in the code: https://git.launchpad.net/cloud-init/tree/cloudinit/config/cc_disk_setup.py#n952

The problem is because `fs_cmd` is a list, not a string, which is
related to: https://bugs.launchpad.net/cloud-init/+bug/1755965

Also the documentation is not accurate, because the example shows a
string, and actually the type of `cmd` in the code is a list: See here:
https://cloudinit.readthedocs.io/en/latest/topics/examples.html#disk-
setup

** Affects: cloud-init
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1801790

Title:
  Crash running custom command on fs_setup

Status in cloud-init:
  New

Bug description:
  Given this cloud-init config:

      fs_setup:
      - cmd:['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sda', '--subvol', 'data:/data']
        label: volumedata
        filesystem: 'btrfs'
        device: '/dev/sda'

  Error log:

      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Writing to /var/lib/cloud/instances/raspberrypi/sem/config_disk_setup - wb: [420] 24 bytes
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] helpers.py[DEBUG]: Running config-disk_setup using lock (<FileLock using file '/var/lib/cloud/instances/raspberrypi/sem/config_disk_setup'>)
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: setting up filesystems: [{'label': 'volumedata', 'filesystem': 'btrfs', 'cmd': ['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sda', '--subvol', 'data:/data'], 'device': '/dev/sda'}, {'label': 'volumedata', 'filesystem': 'btrfs', 'cmd': ['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sdb'], 'device': '/dev/sdb'}]
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating new filesystem.
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking /dev/sda against default devices
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using the raw device to place filesystem volumedata on
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: File system type 'btrfs' with label 'volumedata' will be created on /dev/sda
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Creating fs for /dev/sda took 0.003 seconds
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[WARNING]: Failed during filesystem operation
                                             unsupported operand type(s) for %: 'list' and 'dict'
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Failed during filesystem operation
                                             unsupported operand type(s) for %: 'list' and 'dict'
                                             Traceback (most recent call last):
                                               File "/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py", line 155, in handle
                                                 func=mkfs, args=(definition,))
                                               File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2181, in log_time
                                                 ret = func(*args, **kwargs)
                                               File "/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py", line 917, in mkfs
                                                 'device': device,
                                             TypeError: unsupported operand type(s) for %: 'list' and 'dict'
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating new filesystem.
      Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking /dev/sdb against default devices

  
  Link to the bug in the code: https://git.launchpad.net/cloud-init/tree/cloudinit/config/cc_disk_setup.py#n952

  The problem is because `fs_cmd` is a list, not a string, which is
  related to: https://bugs.launchpad.net/cloud-init/+bug/1755965

  Also the documentation is not accurate, because the example shows a
  string, and actually the type of `cmd` in the code is a list: See
  here: https://cloudinit.readthedocs.io/en/latest/topics/examples.html
  #disk-setup

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1801790/+subscriptions
Follow ups

[Bug 1801790] Re: Crash running custom command on fs_setup
From: Launchpad Bug Tracker, 2019-09-17