← Back to team overview

group.of.nepali.translators team mailing list archive

[Bug 1623570] Re: Azure: cannot start walinux agent (Transaction order is cyclic.)

 

Hello Scott, or anyone else affected,

Accepted walinuxagent into xenial-proposed. The package will build now
and be available at
https://launchpad.net/ubuntu/+source/walinuxagent/2.1.3-0ubuntu4.1 in a
few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed.  Your feedback will aid us getting this update
out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: walinuxagent (Ubuntu Xenial)
       Status: Confirmed => Fix Committed

** No longer affects: cloud-init (Ubuntu Xenial)

-- 
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/1623570

Title:
  Azure: cannot start walinux agent (Transaction order is cyclic.)

Status in walinuxagent package in Ubuntu:
  Fix Released
Status in walinuxagent source package in Xenial:
  Fix Committed

Bug description:
  http://pad.lv/1623570
  ==== Begin SRU Template ====
  [Impact] 
  On Azure (and only on Azure), cloud-init utilizes the walinuxagent
  to do some of the instance initialization.  A change to cloud-init
  to support installation of packages on a systemd system (bug 1576692),
  exposed an issue in the walinuxagent.service file.

  The end result is freshly booted instances on azure with the cloud-init
  fix and the old version of walinuxagent would fail to import ssh keys
  that were provided to the instance.

  [Test Case]
   * Start an instance on Azure
   * upgrade to cloud-init at 0.7.7-31-g65ace7b or better 
     this is in both xenial-proposed and yakkety.
     
   * clean up, so it looks like a first boot.
     sudo sh -c 'rm -Rf /var/lib/cloud /var/log/cloud-init* /var/lib/waagent/*.crt'
   * At this point a reboot would cause failure, and you would see WARN messages in the logs.

   * upgrade walinuxagent with a fix (2.1.5-0ubuntu2 in yakkety)

   * reboot
   * ssh back in, 
     * There should be no WARN in /var/log/cloud-init.log
       grep WARN /var/log/cloud-init.log
     * There should exist a file /var/lib/waagent/*.crt
     * walinuxagent.service should be running
       systemctl status walinuxagent.service

  Note, that in event of failure you can still get into the system by
  providing user-data that populates your ssh keys:
    #!/bin/sh
    ssh-import-id smoser

  [Regression Potential] 
  Changing order of things in boot is scary.  This is no different.
  This would in theory allow walinux-agent to start earlier in the boot.
  In practice, however, it is started by cloud-init.service so it was
  already running that early in the boot.
  ==== End SRU Template ====

  
  When bringing up the Azure datasource in cloud-init.service, cloud-init tries 'service start walinuxagent'.

  That previously worked fine, and the agent would start and then would
  produce the certificate files that cloud-init needed (for ssh keys and
  things).

  I found this when testing SRU for 0.7.7-31-g65ace7b-0ubuntu1~16.04.1
  but it is likely present also in 0.7.7-31-g65ace7b-0ubuntu1 (yakkety)

  Now, however we see a log like:
  Sep 14 14:53:18 smoser0914x [CLOUDINIT] DataSourceAzure.py[DEBUG]: Getting metadata via agent.  hostname=smoser0914x cmd=['service', 'walinuxagent', 'start']
  Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[DEBUG]: Running command hostname with allowed return codes [0] (shell=False, capture=True)
  Sep 14 14:53:18 smoser0914x [CLOUDINIT] DataSourceAzure.py[DEBUG]: invoking agent: ['service', 'walinuxagent', 'start']
  Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[DEBUG]: Running command ['service', 'walinuxagent', 'start'] with allowed return codes [0] (shell=False, capture=True)
  Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[WARNING]: agent command '['service', 'walinuxagent', 'start']' failed.
  Sep 14 14:53:19 smoser0914x [CLOUDINIT] util.py[DEBUG]: agent command '['service', 'walinuxagent', 'start']' failed.
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 145, in get_metadata_from_agent
      invoke_agent(agent_cmd)
    File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 452, in invoke_agent
      util.subp(cmd, shell=(not isinstance(cmd, list)))
    File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1832, in subp
      cmd=args)
  cloudinit.util.ProcessExecutionError: Unexpected error while running command.
  Command: ['service', 'walinuxagent', 'start']
  Exit code: 1
  Reason: -
  Stdout: ''
  Stderr: "
    Failed to start walinuxagent.service: Transaction order is cyclic. See system logs for details.
    See system logs and 'systemctl status walinuxagent.service' for details

  I believe the relevant change is in 34a26f7f
    https://git.launchpad.net/cloud-init/commit/?id=34a26f7f59f2963691e36ca0476bec9fc9ccef63
  That added multi-user.target to the list of After for cloud-init-final.service.

  Related bugs:
   * bug 1576692:  fully support package installation in systemd

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/walinuxagent/+bug/1623570/+subscriptions


References