group.of.nepali.translators team mailing list archive
-
group.of.nepali.translators team
-
Mailing list archive
-
Message #07543
[Bug 1623570] Re: Azure: cannot start walinux agent (Transaction order is cyclic.)
Hello Scott, or anyone else affected,
Accepted walinuxagent into xenial-proposed. The package will build now
and be available at
https://launchpad.net/ubuntu/+source/walinuxagent/2.1.3-0ubuntu4.1 in a
few hours, and then in the -proposed repository.
Please help us by testing this new package. See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed. Your feedback will aid us getting this update
out to other Ubuntu users.
If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed. In either case, details of your testing will help
us make a better decision.
Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
advance!
** Changed in: walinuxagent (Ubuntu Xenial)
Status: Confirmed => Fix Committed
** No longer affects: cloud-init (Ubuntu Xenial)
--
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/1623570
Title:
Azure: cannot start walinux agent (Transaction order is cyclic.)
Status in walinuxagent package in Ubuntu:
Fix Released
Status in walinuxagent source package in Xenial:
Fix Committed
Bug description:
http://pad.lv/1623570
==== Begin SRU Template ====
[Impact]
On Azure (and only on Azure), cloud-init utilizes the walinuxagent
to do some of the instance initialization. A change to cloud-init
to support installation of packages on a systemd system (bug 1576692),
exposed an issue in the walinuxagent.service file.
The end result is freshly booted instances on azure with the cloud-init
fix and the old version of walinuxagent would fail to import ssh keys
that were provided to the instance.
[Test Case]
* Start an instance on Azure
* upgrade to cloud-init at 0.7.7-31-g65ace7b or better
this is in both xenial-proposed and yakkety.
* clean up, so it looks like a first boot.
sudo sh -c 'rm -Rf /var/lib/cloud /var/log/cloud-init* /var/lib/waagent/*.crt'
* At this point a reboot would cause failure, and you would see WARN messages in the logs.
* upgrade walinuxagent with a fix (2.1.5-0ubuntu2 in yakkety)
* reboot
* ssh back in,
* There should be no WARN in /var/log/cloud-init.log
grep WARN /var/log/cloud-init.log
* There should exist a file /var/lib/waagent/*.crt
* walinuxagent.service should be running
systemctl status walinuxagent.service
Note, that in event of failure you can still get into the system by
providing user-data that populates your ssh keys:
#!/bin/sh
ssh-import-id smoser
[Regression Potential]
Changing order of things in boot is scary. This is no different.
This would in theory allow walinux-agent to start earlier in the boot.
In practice, however, it is started by cloud-init.service so it was
already running that early in the boot.
==== End SRU Template ====
When bringing up the Azure datasource in cloud-init.service, cloud-init tries 'service start walinuxagent'.
That previously worked fine, and the agent would start and then would
produce the certificate files that cloud-init needed (for ssh keys and
things).
I found this when testing SRU for 0.7.7-31-g65ace7b-0ubuntu1~16.04.1
but it is likely present also in 0.7.7-31-g65ace7b-0ubuntu1 (yakkety)
Now, however we see a log like:
Sep 14 14:53:18 smoser0914x [CLOUDINIT] DataSourceAzure.py[DEBUG]: Getting metadata via agent. hostname=smoser0914x cmd=['service', 'walinuxagent', 'start']
Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[DEBUG]: Running command hostname with allowed return codes [0] (shell=False, capture=True)
Sep 14 14:53:18 smoser0914x [CLOUDINIT] DataSourceAzure.py[DEBUG]: invoking agent: ['service', 'walinuxagent', 'start']
Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[DEBUG]: Running command ['service', 'walinuxagent', 'start'] with allowed return codes [0] (shell=False, capture=True)
Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[WARNING]: agent command '['service', 'walinuxagent', 'start']' failed.
Sep 14 14:53:19 smoser0914x [CLOUDINIT] util.py[DEBUG]: agent command '['service', 'walinuxagent', 'start']' failed.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 145, in get_metadata_from_agent
invoke_agent(agent_cmd)
File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 452, in invoke_agent
util.subp(cmd, shell=(not isinstance(cmd, list)))
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1832, in subp
cmd=args)
cloudinit.util.ProcessExecutionError: Unexpected error while running command.
Command: ['service', 'walinuxagent', 'start']
Exit code: 1
Reason: -
Stdout: ''
Stderr: "
Failed to start walinuxagent.service: Transaction order is cyclic. See system logs for details.
See system logs and 'systemctl status walinuxagent.service' for details
I believe the relevant change is in 34a26f7f
https://git.launchpad.net/cloud-init/commit/?id=34a26f7f59f2963691e36ca0476bec9fc9ccef63
That added multi-user.target to the list of After for cloud-init-final.service.
Related bugs:
* bug 1576692: fully support package installation in systemd
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/walinuxagent/+bug/1623570/+subscriptions
References