yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1577753] Re: Cloud-init fails of stage init

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Narinder Gupta <1577753@xxxxxxxxxxxxxxxxxx>
Date: Fri, 08 Jul 2016 13:32:41 -0000
Reply-to: Bug 1577753 <1577753@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
[  OK  ] Started Accounts Service.
[    6.186699] cloud-init[552]: Cloud-init v. 0.7.7 running 'init-local' at Fri, 08 Jul 2016 13:30:15 +0000. Up 6.14 seconds.
[    6.188555] cloud-init[552]: 2016-07-08 13:30:15,769 - util.py[WARNING]: failed of stage init-local
[    6.198610] cloud-init[552]: failed run of stage init-local
[    6.199817] cloud-init[552]: ------------------------------------------------------------
[    6.201545] cloud-init[552]: Traceback (most recent call last):
[    6.203028] cloud-init[552]:   File "/usr/bin/cloud-init", line 520, in status_wrapper
[    6.204659] cloud-init[552]:     ret = functor(name, args)
[    6.206211] cloud-init[552]:   File "/usr/bin/cloud-init", line 250, in main_init
[    6.207780] cloud-init[552]:     init.fetch(existing=existing)
[    6.209292] cloud-init[552]:   File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 322, in fetch
[    6.211067] cloud-init[552]:     return self._get_data_source(existing=existing)
[    6.212661] cloud-init[552]:   File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 241, in _get_data_source
[    6.214496] cloud-init[552]:     util.del_file(self.paths.instance_link)
[    6.216048] cloud-init[552]:   File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1567, in del_file
[    6.217827] cloud-init[552]:     raise e
[    6.219214] cloud-init[552]:   File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1564, in del_file
[    6.221006] cloud-init[552]:     os.unlink(path)
[    6.222416] cloud-init[552]: IsADirectoryError: [Errno 21] Is a directory: '/var/lib/cloud/instance'
[    6.224049] cloud-init[552]: ------------------------------------------------------------
[FAILED] Failed to start Initial cloud-init job (pre-networking).
See 'systemctl status cloud-init-local.service' for details.
[  OK  ] Reached target Network (Pre).


** Also affects: opnfv
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1577753

Title:
  Cloud-init fails of stage init

Status in cloud-init:
  New
Status in OPNFV:
  New

Bug description:
  Cloud-init 0.77 on Ubuntu 16.04 
  seems not to be able to connect to the OpenStack Metadata service and fails, 
  although the Metadata service is available itself, see log here:

  root@ubuntu:/etc/network# cloud-init  --debug init
  2016-05-03 12:25:04,855 - handlers.py[DEBUG]: start: init-network: searching for network datasources
  2016-05-03 12:25:04,855 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
  2016-05-03 12:25:04,855 - util.py[DEBUG]: Read 16 bytes from /proc/uptime
  2016-05-03 12:25:04,856 - util.py[DEBUG]: Reading from /var/lib/cloud/data/status.json (quiet=False)
  2016-05-03 12:25:04,856 - util.py[DEBUG]: Read 548 bytes from /var/lib/cloud/data/status.json
  2016-05-03 12:25:04,857 - util.py[DEBUG]: Creating symbolic link from '/run/cloud-init/status.json' => '../../var/lib/cloud/data/status.json'
  2016-05-03 12:25:04,858 - util.py[DEBUG]: Attempting to remove /run/cloud-init/status.json
  2016-05-03 12:25:04,858 - util.py[DEBUG]: Running command ['systemd-detect-virt', '--quiet', '--container'] with allowed return codes [0] (shell=False, capture=True)
  2016-05-03 12:25:04,861 - util.py[DEBUG]: Running command ['running-in-container'] with allowed return codes [0] (shell=False, capture=True)
  2016-05-03 12:25:04,862 - util.py[DEBUG]: Running command ['lxc-is-container'] with allowed return codes [0] (shell=False, capture=True)
  2016-05-03 12:25:04,864 - util.py[DEBUG]: Reading from /proc/1/environ (quiet=False)
  2016-05-03 12:25:04,864 - util.py[DEBUG]: Read 110 bytes from /proc/1/environ
  2016-05-03 12:25:04,864 - util.py[DEBUG]: Reading from /proc/self/status (quiet=False)
  2016-05-03 12:25:04,865 - util.py[DEBUG]: Read 896 bytes from /proc/self/status
  2016-05-03 12:25:04,865 - util.py[DEBUG]: Reading from /proc/cmdline (quiet=False)
  2016-05-03 12:25:04,865 - util.py[DEBUG]: Read 64 bytes from /proc/cmdline
  2016-05-03 12:25:04,866 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
  2016-05-03 12:25:04,866 - util.py[DEBUG]: Read 16 bytes from /proc/uptime
  2016-05-03 12:25:04,866 - templater.py[WARNING]: Cheetah not available as the default renderer for unknown template, reverting to the basic renderer.
  2016-05-03 12:25:04,867 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg (quiet=False)
  2016-05-03 12:25:04,867 - util.py[DEBUG]: Read 3011 bytes from /etc/cloud/cloud.cfg
  2016-05-03 12:25:04,867 - util.py[DEBUG]: Attempting to load yaml from string of length 3011 with allowed root types (<class 'dict'>,)
  2016-05-03 12:25:04,887 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/90_dpkg.cfg (quiet=False)
  2016-05-03 12:25:04,887 - util.py[DEBUG]: Read 197 bytes from /etc/cloud/cloud.cfg.d/90_dpkg.cfg
  2016-05-03 12:25:04,887 - util.py[DEBUG]: Attempting to load yaml from string of length 197 with allowed root types (<class 'dict'>,)
  2016-05-03 12:25:04,889 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/05_logging.cfg (quiet=False)
  2016-05-03 12:25:04,890 - util.py[DEBUG]: Read 1910 bytes from /etc/cloud/cloud.cfg.d/05_logging.cfg
  2016-05-03 12:25:04,890 - util.py[DEBUG]: Attempting to load yaml from string of length 1910 with allowed root types (<class 'dict'>,)
  2016-05-03 12:25:04,897 - cloud-init[DEBUG]: Closing stdin
  2016-05-03 12:25:04,898 - util.py[DEBUG]: Redirecting <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'> to | tee -a /var/log/cloud-init-output.log
  2016-05-03 12:25:04,899 - util.py[DEBUG]: Redirecting <_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'> to | tee -a /var/log/cloud-init-output.log
  2016-05-03 12:25:04,900 - cloud-init[DEBUG]: Logging being reset, this logger may no longer be active shortly
  Cloud-init v. 0.7.7 running 'init' at Tue, 03 May 2016 12:25:04 +0000. Up 4339.24 seconds.
  ci-info: ++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
  ci-info: +--------+------+------------------------------+---------------+-------+-------------------+
  ci-info: | Device |  Up  |           Address            |      Mask     | Scope |     Hw-Address    |
  ci-info: +--------+------+------------------------------+---------------+-------+-------------------+
  ci-info: |   lo   | True |          127.0.0.1           |   255.0.0.0   |   .   |         .         |
  ci-info: |   lo   | True |           ::1/128            |       .       |  host |         .         |
  ci-info: | ens32  | True |         192.168.0.15         | 255.255.255.0 |   .   | fa:16:3e:40:92:3f |
  ci-info: | ens32  | True | fe80::f816:3eff:fe40:923f/64 |       .       |  link | fa:16:3e:40:92:3f |
  ci-info: +--------+------+------------------------------+---------------+-------+-------------------+
  ci-info: +++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++
  ci-info: +-------+-------------+-------------+---------------+-----------+-------+
  ci-info: | Route | Destination |   Gateway   |    Genmask    | Interface | Flags |
  ci-info: +-------+-------------+-------------+---------------+-----------+-------+
  ci-info: |   0   |   0.0.0.0   | 192.168.0.1 |    0.0.0.0    |   ens32   |   UG  |
  ci-info: |   1   | 192.168.0.0 |   0.0.0.0   | 255.255.255.0 |   ens32   |   U   |
  ci-info: +-------+-------------+-------------+---------------+-----------+-------+
  2016-05-03 12:25:04,920 - util.py[WARNING]: failed of stage init
  failed run of stage init
  ------------------------------------------------------------
  Traceback (most recent call last):
    File "/usr/bin/cloud-init", line 520, in status_wrapper
      ret = functor(name, args)
    File "/usr/bin/cloud-init", line 250, in main_init
      init.fetch(existing=existing)
    File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 322, in fetch
      return self._get_data_source(existing=existing)
    File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 241, in _get_data_source
      util.del_file(self.paths.instance_link)
    File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1567, in del_file
      raise e
    File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1564, in del_file
      os.unlink(path)
  IsADirectoryError: [Errno 21] Is a directory: '/var/lib/cloud/instance'
  ------------------------------------------------------------
  root@ubuntu:/etc/network# systemctl status cloud-init
  ● cloud-init.service - Initial cloud-init job (metadata service crawler)
     Loaded: loaded (/lib/systemd/system/cloud-init.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Tue 2016-05-03 11:27:14 UTC; 57min ago
    Process: 773 ExecStart=/usr/bin/cloud-init init (code=exited, status=1/FAILURE)
   Main PID: 773 (code=exited, status=1/FAILURE)

  May 03 11:27:14 ubuntu cloud-init[773]:   File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1567, in del_file
  May 03 11:27:14 ubuntu cloud-init[773]:     raise e
  May 03 11:27:14 ubuntu cloud-init[773]:   File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1564, in del_file
  May 03 11:27:14 ubuntu cloud-init[773]:     os.unlink(path)
  May 03 11:27:14 ubuntu cloud-init[773]: IsADirectoryError: [Errno 21] Is a directory: '/var/lib/cloud/instance'
  May 03 11:27:14 ubuntu cloud-init[773]: ------------------------------------------------------------
  May 03 11:27:14 ubuntu systemd[1]: cloud-init.service: Main process exited, code=exited, status=1/FAILURE
  May 03 11:27:14 ubuntu systemd[1]: Failed to start Initial cloud-init job (metadata service crawler).
  May 03 11:27:14 ubuntu systemd[1]: cloud-init.service: Unit entered failed state.
  May 03 11:27:14 ubuntu systemd[1]: cloud-init.service: Failed with result 'exit-code'.
  root@ubuntu:/etc/network#

  meta-data/root@ubuntu:/etc/network# curl http://169.254.169.254/2009-04-04/meta-data/instance-id
           i-00000220
  root@ubuntu:/etc/network#

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1577753/+subscriptions
References

[Bug 1577753] [NEW] Cloud-init fails of stage init
From: krath, 2016-05-03