← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #65872

[Bug 1704466] [NEW] Containers failing to properly set interface IP addresses during juju deploy

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

During a customer deploy.  Some containers (different ones for different
services on each deploy) will not get their ip addresses.  The
interfaces for the containers are being configured, but no IP address
assigned to any of the multiple interfaces.

Doing a juju remove-unit and then add-unit to the same lxd host works
fine and the interfaces and machine come up as expected.  Random issue.

This is at a secure facility so pulling logs is hard, but this is from
cloud-init.log on the container failing.  the only error message I could
find.


2017-07-14 17:45:58,707 - util.py[DEBUG]: Read 3033 bytes from /etc/cloud/cloud.cfg
2017-07-14 17:45:58,707 - util.py[DEBUG]: Attempting to load yaml from string of length 3033 with allowed root types (<class 'dict'>,)
2017-07-14 17:45:58,722 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/90_dpkg.cfg (quiet=False)
2017-07-14 17:45:58,722 - util.py[DEBUG]: Read 220 bytes from /etc/cloud/cloud.cfg.d/90_dpkg.cfg
2017-07-14 17:45:58,722 - util.py[DEBUG]: Attempting to load yaml from string of length 220 with allowed root types (<class 'dict'>,)
2017-07-14 17:45:58,724 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/05_logging.cfg (quiet=False)
2017-07-14 17:45:58,724 - util.py[DEBUG]: Read 2057 bytes from /etc/cloud/cloud.cfg.d/05_logging.cfg
2017-07-14 17:45:58,724 - util.py[DEBUG]: Attempting to load yaml from string of length 2057 with allowed root types (<class 'dict'>,)
2017-07-14 17:45:58,728 - util.py[DEBUG]: Reading from /run/cloud-init/cloud.cfg (quiet=False)
2017-07-14 17:45:58,728 - util.py[DEBUG]: Read 48 bytes from /run/cloud-init/cloud.cfg
2017-07-14 17:45:58,728 - util.py[DEBUG]: Attempting to load yaml from string of length 48 with allowed root types (<class 'dict'>,)
2017-07-14 17:45:58,729 - util.py[DEBUG]: Attempting to load yaml from string of length 0 with allowed root types (<class 'dict'>,)
2017-07-14 17:45:58,729 - util.py[DEBUG]: load_yaml given empty string, returning default
2017-07-14 17:45:58,730 - stages.py[DEBUG]: applying net config names for {'network': {'config': 'disabled'}}
2017-07-14 17:45:58,731 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.ubuntu.Distro'>
2017-07-14 17:45:58,731 - __init__.py[DEBUG]: no interfaces to rename
2017-07-14 17:45:58,731 - stages.py[INFO]: Applying network configuration from ds bringup=True: {'network': {'config': 'disabled'}}
2017-07-14 17:45:58,731 - __init__.py[DEBUG]: Selected renderer 'eni' from priority list: None
2017-07-14 17:45:58,731 - util.py[WARNING]: failed stage init
2017-07-14 17:45:58,731 - util.py[DEBUG]: failed stage init
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 648, in status_wrapper
    ret = functor(name, args)
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 365, in main_init
    init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 649, in apply_network_config
    return self.distro.apply_network_config(netcfg, bring_up=bring_up)
  File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 163, in apply_network_config
    dev_names = self._write_network_config(netconfig)
  File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 89, in _write_network_config
    return self._supported_write_network_config(netconfig)
  File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 82, in _supported_write_network_config
    renderer.render_network_config(network_config=network_config)
  File "/usr/lib/python3/dist-packages/cloudinit/net/renderer.py", line 47, in render_network_config
    network_state=parse_net_config_data(network_config), target=target)
  File "/usr/lib/python3/dist-packages/cloudinit/net/eni.py", line 446, in render_network_state
    util.write_file(fpeni, header + self._render_interfaces(network_state))
  File "/usr/lib/python3/dist-packages/cloudinit/net/eni.py", line 403, in _render_interfaces
    for iface in network_state.iter_interfaces():
AttributeError: 'NoneType' object has no attribute 'iter_interfaces'
2017-07-14 17:45:58,733 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
2017-07-14 17:45:58,735 - util.py[DEBUG]: Read 8 bytes from /proc/uptime
2017-07-14 17:45:58,735 - util.py[DEBUG]: cloud-init mode 'init' took 0.153 seconds (0.00)
2017-07-14 17:45:58,735 - handlers.py[DEBUG]: finish: init-network: SUCCESS: searching for network datasources

cloud-init version 0.7.9-113 on xenial

maas 2.2.0 and juju 2.2.1

** Affects: cloud-init
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1704466

Title:
  Containers failing to properly set interface IP addresses during juju
  deploy

Status in cloud-init:
  New

Bug description:
  During a customer deploy.  Some containers (different ones for
  different services on each deploy) will not get their ip addresses.
  The interfaces for the containers are being configured, but no IP
  address assigned to any of the multiple interfaces.

  Doing a juju remove-unit and then add-unit to the same lxd host works
  fine and the interfaces and machine come up as expected.  Random
  issue.

  This is at a secure facility so pulling logs is hard, but this is from
  cloud-init.log on the container failing.  the only error message I
  could find.

  
  2017-07-14 17:45:58,707 - util.py[DEBUG]: Read 3033 bytes from /etc/cloud/cloud.cfg
  2017-07-14 17:45:58,707 - util.py[DEBUG]: Attempting to load yaml from string of length 3033 with allowed root types (<class 'dict'>,)
  2017-07-14 17:45:58,722 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/90_dpkg.cfg (quiet=False)
  2017-07-14 17:45:58,722 - util.py[DEBUG]: Read 220 bytes from /etc/cloud/cloud.cfg.d/90_dpkg.cfg
  2017-07-14 17:45:58,722 - util.py[DEBUG]: Attempting to load yaml from string of length 220 with allowed root types (<class 'dict'>,)
  2017-07-14 17:45:58,724 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/05_logging.cfg (quiet=False)
  2017-07-14 17:45:58,724 - util.py[DEBUG]: Read 2057 bytes from /etc/cloud/cloud.cfg.d/05_logging.cfg
  2017-07-14 17:45:58,724 - util.py[DEBUG]: Attempting to load yaml from string of length 2057 with allowed root types (<class 'dict'>,)
  2017-07-14 17:45:58,728 - util.py[DEBUG]: Reading from /run/cloud-init/cloud.cfg (quiet=False)
  2017-07-14 17:45:58,728 - util.py[DEBUG]: Read 48 bytes from /run/cloud-init/cloud.cfg
  2017-07-14 17:45:58,728 - util.py[DEBUG]: Attempting to load yaml from string of length 48 with allowed root types (<class 'dict'>,)
  2017-07-14 17:45:58,729 - util.py[DEBUG]: Attempting to load yaml from string of length 0 with allowed root types (<class 'dict'>,)
  2017-07-14 17:45:58,729 - util.py[DEBUG]: load_yaml given empty string, returning default
  2017-07-14 17:45:58,730 - stages.py[DEBUG]: applying net config names for {'network': {'config': 'disabled'}}
  2017-07-14 17:45:58,731 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.ubuntu.Distro'>
  2017-07-14 17:45:58,731 - __init__.py[DEBUG]: no interfaces to rename
  2017-07-14 17:45:58,731 - stages.py[INFO]: Applying network configuration from ds bringup=True: {'network': {'config': 'disabled'}}
  2017-07-14 17:45:58,731 - __init__.py[DEBUG]: Selected renderer 'eni' from priority list: None
  2017-07-14 17:45:58,731 - util.py[WARNING]: failed stage init
  2017-07-14 17:45:58,731 - util.py[DEBUG]: failed stage init
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 648, in status_wrapper
      ret = functor(name, args)
    File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 365, in main_init
      init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
    File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 649, in apply_network_config
      return self.distro.apply_network_config(netcfg, bring_up=bring_up)
    File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 163, in apply_network_config
      dev_names = self._write_network_config(netconfig)
    File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 89, in _write_network_config
      return self._supported_write_network_config(netconfig)
    File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 82, in _supported_write_network_config
      renderer.render_network_config(network_config=network_config)
    File "/usr/lib/python3/dist-packages/cloudinit/net/renderer.py", line 47, in render_network_config
      network_state=parse_net_config_data(network_config), target=target)
    File "/usr/lib/python3/dist-packages/cloudinit/net/eni.py", line 446, in render_network_state
      util.write_file(fpeni, header + self._render_interfaces(network_state))
    File "/usr/lib/python3/dist-packages/cloudinit/net/eni.py", line 403, in _render_interfaces
      for iface in network_state.iter_interfaces():
  AttributeError: 'NoneType' object has no attribute 'iter_interfaces'
  2017-07-14 17:45:58,733 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
  2017-07-14 17:45:58,735 - util.py[DEBUG]: Read 8 bytes from /proc/uptime
  2017-07-14 17:45:58,735 - util.py[DEBUG]: cloud-init mode 'init' took 0.153 seconds (0.00)
  2017-07-14 17:45:58,735 - handlers.py[DEBUG]: finish: init-network: SUCCESS: searching for network datasources

  cloud-init version 0.7.9-113 on xenial

  maas 2.2.0 and juju 2.2.1

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1704466/+subscriptions
  Thread Previous • Date Previous • • Thread Next