yahoo-eng-team team mailing list archive

[Launchpad Bug Tracker <1684349@xxxxxxxxxxxxxxxxxx>]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-
0ubuntu1~17.04.1

---------------
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~17.04.1) zesty-proposed; urgency=medium

  * New upstream snapshot.
    - net: fix reading and rendering addresses in cidr format.
      [Dimitri John Ledkov] (LP: #1689346, #1684349)
    - disk_setup: udev settle before attempting partitioning or fs creation.
      (LP: #1692093)
    - GCE: Update the attribute used to find instance SSH keys.
      [Daniel Watkins] (LP: #1693582)
    - nplan: For bonds, allow dashed or underscore names of keys.
      [Dimitri John Ledkov] (LP: #1690480)
    - tests: python2.6: fix unit tests usage of assertNone and format.
    - tests: update docstring on test_configured_list_with_none
    - fix tools/ds-identify to not write None twice.
    - tox/build: do not package depend on style requirements.
    - tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration.  [Andreas Karis]
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - tests: Fix unittest bug in ntp tests. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - tests: fix cloudstack unit tests to avoid accessing
      /var/lib/NetworkManager [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present.
      [Chris Brinker] (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - make deb: Add devscripts dependency for make deb.
      Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network configuration defaults policy and formats.
      [Ryan Harper]
    - doc: Fix name of "uri" key in docs for "cc_apt_configure" module
      [Felix Dreissig]
    - tests: Enable artful in integration tests [Joshua Powers]

 -- Scott Moser <smoser@xxxxxxxxxx>  Fri, 26 May 2017 16:14:09 -0400

** Changed in: cloud-init (Ubuntu Zesty)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1684349

Title:
   mask2cidr error with integer value - argument of type 'int' is not
  iterable

Status in cloud-init:
  Fix Committed
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Xenial:
  Fix Released
Status in cloud-init source package in Yakkety:
  Fix Released
Status in cloud-init source package in Zesty:
  Fix Released

Bug description:
  === Begin SRU Template ===
  [Impact]
  On Openstack instances, when rendering sysconfig output, cloud-init
  would stacktrace due to a TypeError.
  This affects runtime only when rendering sysconfig networking, which
  is what is used on CentOS and RedHat systems.

  [Test Case]
  The basic idea below is:
   a.) launch an instance with proposed version of cloud-init.
   b.) inside instance, get cloud-init's network rendering tool from trunk
   c.) run the rendering tool against a config that failed before.
   d.) check rendered netplan config to verify it has the correct format.
       The failed output would have 'addresses' with a format like:
       172.19.1.34/255.255.255.0
       The expected output would be 'cidr' format:
       172.19.1.34/24

  ## launch an instance.
  $ release=xenial
  $ ref=$release-proposed
  $ lxc-proposed-snapshot --proposed --publish $release $ref
  $ lxc launch $ref $name
  $ lxc exec $name

  ## get render tool
  % wget https://git.launchpad.net/~cloud-init-dev/cloud-init/plain/tools/net-convert.py -O net-convert.py

  ## write the network_data.json
  % cat > simple-ipv6.yaml <<EOF
  version: 1
  config:
    - type: physical
      name: eth0
      subnets:
       - type: static
         address: "2000:192:168::5"
         netmask: 64
         routes:
          - netmask: 0
            gateway: "2000:192:168::1"
            network: "::"
  EOF

  ## run the converter
  % ./net-convert.py --network-data=simple-ipv6.yaml \
       --kind=yaml --output-kind=eni --directory=out.d

  ## check the output
  % cat out.d/etc/network/interfaces
  auto lo
  iface lo inet loopback

  auto eth0
  iface eth0 inet6 static
      address 2000:192:168::5
      netmask 64
      post-up route add -A inet6 default gw 2000:192:168::1 || true
      pre-down route del -A inet6 default gw 2000:192:168::1 || true

  ## show the cloud-init versions
  % dpkg-query --show cloud-init
  ...

  [Regression Potential]
  The fix here was just to make a common networking method accept
  a string input as intended rather than only an integer.

  The common code changes could shake out other failures in the networking
  path.

  [Other Info]
  Upstream commit at
    https://git.launchpad.net/cloud-init/commit/?id=16a7302f6a

  lxc-proposed-snapshot is
    https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bin/lxc-proposed-snapshot
  It publishes an image to lxd with proposed enabled and cloud-init upgraded.
  === End SRU Template ===

  mask2cidr error with integer value - argument of type 'int' is not
  iterable

  ~~~
  def mask2cidr(mask):
      if ':' in str(mask):
          return ipv6mask2cidr(mask)
      elif '.' in mask:
          return ipv4mask2cidr(mask)
      else:
          return mask
  ~~~

  is not type safe. It tries to take into account that this can be a
  prefix (so it does not contain ':' not '.' and then return mask. The
  problem is that if mask is an integer, then this returns:

  ~~~
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/cloudinit/cmd/main.py", line 513, in status_wrapper
      ret = functor(name, args)
    File "/usr/lib/python2.7/site-packages/cloudinit/cmd/main.py", line 269, in main_init
      init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
    File "/usr/lib/python2.7/site-packages/cloudinit/stages.py", line 641, in apply_network_config
      return self.distro.apply_network_config(netcfg, bring_up=bring_up)
    File "/usr/lib/python2.7/site-packages/cloudinit/distros/__init__.py", line 150, in apply_network_config
      dev_names = self._write_network_config(netconfig)
    File "/usr/lib/python2.7/site-packages/cloudinit/distros/rhel.py", line 59, in _write_network_config
      ns = parse_net_config_data(netconfig)
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 32, in parse_net_config_data
      nsi.parse_config(skip_broken=skip_broken)
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 205, in parse_config
      handler(self, command)
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 78, in decorator
      return func(self, command, *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 239, in handle_physical
      subnet['netmask'] = mask2cidr(subnet['netmask'])
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 441, in mask2cidr
      elif '.' in mask:
  ~~~

  Made a modification to the code to troubleshoot this:
  ~~~
         # convert subnet ipv6 netmask to cidr as needed
          subnets = command.get('subnets')
          print subnets
          if subnets:
              for subnet in subnets:
                  if subnet['type'] == 'static':
                      if 'netmask' in subnet and ':' in subnet['address']:
                          subnet['netmask'] = mask2cidr(subnet['netmask'])
                          for route in subnet.get('routes', []):
                              if 'netmask' in route:
                                  route['netmask'] = mask2cidr(route['netmask'])
  ~~~

  This error can be hit on RHEL when running the following 2x (don't
  know why 2x):

   rm -Rf /var/lib/cloud/data/*  ; cloud-init --force init

  On the second run, this will be returned:
  ~~~
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/cloudinit/cmd/main.py", line 513, in status_wrapper
      ret = functor(name, args)
    File "/usr/lib/python2.7/site-packages/cloudinit/cmd/main.py", line 269, in main_init
      init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
    File "/usr/lib/python2.7/site-packages/cloudinit/stages.py", line 641, in apply_network_config
      return self.distro.apply_network_config(netcfg, bring_up=bring_up)
    File "/usr/lib/python2.7/site-packages/cloudinit/distros/__init__.py", line 150, in apply_network_config
      dev_names = self._write_network_config(netconfig)
    File "/usr/lib/python2.7/site-packages/cloudinit/distros/rhel.py", line 59, in _write_network_config
      ns = parse_net_config_data(netconfig)
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 32, in parse_net_config_data
      nsi.parse_config(skip_broken=skip_broken)
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 205, in parse_config
      handler(self, command)
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 78, in decorator
      return func(self, command, *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 239, in handle_physical
      subnet['netmask'] = mask2cidr(subnet['netmask'])
    File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", line 441, in mask2cidr
      elif '.' in mask:
  TypeError: argument of type 'int' is not iterable
  ------------------------------------------------------------
  [{u'routes': [{u'netmask': u'0.0.0.0', u'network': u'0.0.0.0', u'gateway': u'192.168.0.1'}], u'netmask': u'255.255.255.0', u'type': 'static', 'ipv4': True, 'address': u'192.168.0.11'}, {u'routes': [{u'netmask': 0, u'network': u'::', u'gateway': u'2000:192:168::1'}], u'netmask': 64, 'ipv6': True, u'type': 'static', 'address': u'2000:192:168::4'}]
  ~~~

  not the `u'netmask': 64` integer

  This can be fixed by changing the code to:
  ~~~
  def mask2cidr(mask):
      if ':' in str(mask):
          return ipv6mask2cidr(mask)
      elif '.' in str(mask):
          return ipv4mask2cidr(mask)
      else:
          return mask
  ~~~

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1684349/+subscriptions