group.of.nepali.translators team mailing list archive
-
group.of.nepali.translators team
-
Mailing list archive
-
Message #35133
[Bug 1693361] Re: cloud-init sometimes fails on dpkg lock due to concurrent apt-daily.service execution
** Changed in: apt
Status: Confirmed => Fix Released
--
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/1693361
Title:
cloud-init sometimes fails on dpkg lock due to concurrent apt-
daily.service execution
Status in APT:
Fix Released
Status in cloud-init:
Fix Released
Status in apt package in Ubuntu:
Invalid
Status in cloud-init package in Ubuntu:
Fix Released
Status in cloud-init source package in Xenial:
Fix Released
Status in cloud-init source package in Yakkety:
Won't Fix
Status in cloud-init source package in Zesty:
Fix Released
Status in cloud-init source package in Artful:
Fix Released
Bug description:
=== Begin SRU Template ===
[Impact]
A cloud-config that contains packages to install (see below) or
'package_upgrade' will run 'apt-get update'. That can sometimes fail as a
result of contention with the apt-daily.service that updates that information.
Cloud-config showing the problem is just like:
$ cat my.yaml
#cloud-config
packages: ['hello']
[Test Case]
lxc-proposed-snapshot is
https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bin/lxc-proposed-snapshot
It publishes an image to lxd with proposed enabled and cloud-init upgraded.
a.) launch an instance with proposed version of cloud-init and some user-data.
This is platform independent. The test case demonstrates lxd.
$ printf "%s\n%s\n%s\n" "#cloud-config" "packages: ['hello']" \
"package_upgrade: true" > config.yaml
$ release=xenial
$ ref=proposed-$release
$ ./lxc-proposed-snapshot --proposed --publish $release $ref;
b.) start the instance
$ name=$release-1693361
$ lxc launch my-xenial "--config=user.user-data=$(cat config.yaml)
$ sleep 1
$ lxc exec $name -- tail -f /var/log/cloud-init.log /var/log/cloud-init-output.log
# watch this boot.
c.) Look for evidence of systemd failure
journalctl -o short-precise | grep -i break
journalctl -o short-precise | grep -i order
[Regression Potential]
Regression chance here is low. Its possible that ordering loops
could occur. When that does happen, journalctl will mention it. Unfortunately
in such cases systemd somewhat randomly picks a service to kil so behavior
is somewhat undefined.
[Other Info]
Upstream commit at
https://git.launchpad.net/cloud-init/commit/?id=11121fe4
=== End SRU Template ===
apt-daily is now a systemd service rather than being invoked by
cron.daily. If one builds a custom AMI it is possible that the apt-
daily.timer will fire during boot. This can fire at the same time
cloud-init is running and if cloud-init loses the race the invocation
of apt (e.g. use of "packages:" in the config) will fail.
There is a lot of discussion online about this change to apt-daily
(e.g. unattended upgrades happening during business hours, delaying
boot, etc.) and discussion of potential systemd changes regarding
timers firing during boot (c.f.
https://github.com/systemd/systemd/issues/5659).
While it would be better to solve this in apt itself, I suggest that
cloud-init be defensive when calling apt and implement some retry
mechanism.
Various instances of people running into this issue:
https://github.com/chef/bento/issues/609
https://clusterhq.atlassian.net/browse/FLOC-4486
https://github.com/boxcutter/ubuntu/issues/73
https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image
To manage notifications about this bug go to:
https://bugs.launchpad.net/apt/+bug/1693361/+subscriptions