yahoo-eng-team team mailing list archive

[Jim Browne <1693361@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

apt-daily is now a systemd service rather than being invoked by
cron.daily.  If one builds a custom AMI it is possible that the apt-
daily.timer will fire during boot.  This can fire at the same time
cloud-init is running and if cloud-init loses the race the invocation of
apt (e.g. use of "packages:" in the config) will fail.

There is a lot of discussion online about this change to apt-daily (e.g.
unattended upgrades happening during business hours, delaying boot,
etc.) and discussion of potential systemd changes regarding timers
firing during boot (c.f.
https://github.com/systemd/systemd/issues/5659).

While it would be better to solve this in apt itself, I suggest that
cloud-init be defensive when calling apt and implement some retry
mechanism.

Various instances of people running into this issue:

https://github.com/chef/bento/issues/609
https://clusterhq.atlassian.net/browse/FLOC-4486
https://github.com/boxcutter/ubuntu/issues/73
https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image

** Affects: cloud-init
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1693361

Title:
  cloud-init sometimes fails on dpkg lock due to concurrent apt-
  daily.service execution

Status in cloud-init:
  New

Bug description:
  apt-daily is now a systemd service rather than being invoked by
  cron.daily.  If one builds a custom AMI it is possible that the apt-
  daily.timer will fire during boot.  This can fire at the same time
  cloud-init is running and if cloud-init loses the race the invocation
  of apt (e.g. use of "packages:" in the config) will fail.

  There is a lot of discussion online about this change to apt-daily
  (e.g. unattended upgrades happening during business hours, delaying
  boot, etc.) and discussion of potential systemd changes regarding
  timers firing during boot (c.f.
  https://github.com/systemd/systemd/issues/5659).

  While it would be better to solve this in apt itself, I suggest that
  cloud-init be defensive when calling apt and implement some retry
  mechanism.

  Various instances of people running into this issue:

  https://github.com/chef/bento/issues/609
  https://clusterhq.atlassian.net/browse/FLOC-4486
  https://github.com/boxcutter/ubuntu/issues/73
  https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1693361/+subscriptions