yahoo-eng-team team mailing list archive

[Scott Moser <smoser@xxxxxxxxxx>]

This is fixed in cloud-init 0.7.9.

** Also affects: cloud-init
   Importance: Undecided
       Status: New

** Changed in: cloud-init
       Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1623868

Title:
  cloud-final.service does not run due to dependency cycle

Status in cloud-init:
  Fix Released
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Xenial:
  Fix Released

Bug description:
  ==== Begin SRU Template ====
  [Impact]
  As part of the change in bug 1576692, we made cloud-final.target
  run After multi-user.target.  That created a dependency loop between
  cloud-init.target and multi-user.target and cloud-final.target.

  Most of the time systemd would break that loop by dropping
  cloud-init.target.  But sometimes, it would break the loop by dropping
  cloud-final.target, which would mean that user scripts do not run
  and generally cloud-init doesn't finish.

  [Test Case]
  ## Failure in a xenial image can only be reproduced by
  ## patching an image up to the previous xenial-proposed upload
  ## (0.7.7-31-g65ace7b-0ubuntu1~16.04.1), then cleaning it
  ## and then restarting.  We will focus on verifying there is not
  ## a problem.

  ## Launch an instance and patch it up to xenial-proposed
  $ release=xenial
  $ name=x1
  $ lxc launch ubuntu-daily:$release $name

  # wait for it to boot
  $ while ! lxc exec $name -- [ -e /run/cloud-init/result.json ]; do sleep 1; done

  ## Now update container, clean and reboot to show first boot
  $ lxc exec $name -- sh -c '
      p=/etc/apt/sources.list.d/proposed.list
      echo deb http://archive.ubuntu.com/ubuntu xenial-proposed main > "$p" &&
      apt-get update -q && apt-get -qy install cloud-init'
  $ lxc exec $name -- sh -c '
      cd /var/lib/cloud && for d in *; do [ "$d" = "seed" ] || rm -Rf "$d"; done
      rm -Rf /var/log/cloud-init*'

  ## This is like first boot now.
  $ lxc exec $name reboot
  $ while ! lxc exec $name -- [ -e /run/cloud-init/result.json ]; do sleep 1; done

  ## The services should show active
  $ lxc exec $name -- journalctl | grep Break || echo Good, no breaks
  $ lxc exec $name -- systemctl --no-pager status cloud-final.service
  $ lxc exec $name -- systemctl --no-pager status cloud-init.target

  [Regression Potential]
  Playing with boot order can cause problems.  Regression would be around
  some targets not running.   On a booted system this would show itself inx
    journalctl | grep -i Break
  or
    journalctl | grep -i ordering

  ==== End SRU Template ====

  With current yakkety cloud images (at least in Scalingstack), I often
  run into this dependency cycle at boot:

  Sep 15 09:28:51 ubuntu systemd[1]: cloud-init.target: Found ordering cycle on cloud-init.target/start
  Sep 15 09:28:51 ubuntu systemd[1]: cloud-init.target: Found dependency on cloud-final.service/start
  Sep 15 09:28:51 ubuntu systemd[1]: cloud-init.target: Found dependency on multi-user.target/start
  Sep 15 09:28:51 ubuntu systemd[1]: cloud-init.target: Found dependency on cloud-init.target/start
  Sep 15 09:28:51 ubuntu systemd[1]: cloud-init.target: Breaking ordering cycle by deleting job cloud-final.service/start
  Sep 15 09:28:51 ubuntu systemd[1]: cloud-final.service: Job cloud-final.service/start deleted to break ordering cycle starting with cloud-init.target/start

  ● cloud-final.service - Execute cloud user/final scripts
     Loaded: loaded (/lib/systemd/system/cloud-final.service; enabled; vendor preset: enabled)
     Active: inactive (dead)

  Thus /var/lib/cloud/instance/boot-finished never gets written and thus
  waiting for an instance to init just times out.

  This is with the most recent https://launchpad.net/ubuntu/+source
  /cloud-init/0.7.7-31-g65ace7b-0ubuntu1

  Related bugs:
   * bug 1576692: fully support package installation in systemd

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1623868/+subscriptions