touch-packages team mailing list archive

[Adam Conrad <adconrad@xxxxxxx>]

Tested in production, and verified that it resolves the crash.

** Tags removed: verification-needed
** Tags added: verification-done

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to upstart in Ubuntu.
https://bugs.launchpad.net/bugs/1430403

Title:
  [SRU] ubuntu-touch livefs builds kill upstart in host

Status in upstart package in Ubuntu:
  In Progress
Status in upstart source package in Precise:
  Fix Released

Bug description:
  = Summary =

  The version of Upstart in precise is affected by a bug in the way that
  ".override" [1] file handling is performed.

  If a job has an override file ("/etc/init/*.override") and that
  override file is deleted before the corresponding job configuration
  file ("/etc/init/*.conf"), there is a possibility of a crash.

  == Explanation ==

  When a "/etc/init/*.override" file is deleted, Upstart will
  automatically detect this and reload the corresponding
  "/etc/init/*.conf" file.

  However, if the ".conf" file (which the ".override" file corresponded
  to) is deleted at the same time Upstart attempts to read the ".conf"
  file, an assertion failure could result.

  = Code Specifics =

  The erroneous function is "conf_delete_handler()" which is called
  whenever a file Upstart is watching gets deleted. The end of this
  function calls conf_reload_path() but although it logs an error
  message, it does not consume the error object that gets raised when
  conf_reload_path() fails.

  = Affected Releases =

  This bug is only present in precise:

  - Upstart override handling was introduced in Upstart v1.3:
  - Precise currently uses Upstart 1.5-0ubuntu7.2 (and hence is affected).
  - Lucid currently uses Upstart 0.6.5-8 (hence, not affected).
  - Trusty and Vivid use much newer versions of the Upstart which no
    longer contain the problematic code.

  = Fix =

  The fix is simply to have conf_delete_handler() consume the error
  object (by freeing it) when conf_reload_path() fails.

  = Test Case =

  A reliable test case is unfortunately not possible to create, since the problem comes down to Upstart racing with the deletion of the ".conf"
  file.

  However, the patch is small and it can be seen that every other
  failing call to conf_reload_path() free's the resulting error object.

  = Workarounds =

  The problem is only manifested if the ".conf" and the ".override" file
  get deleted one after another, with the ".override" file being deleted
  first. This implies the following work-arounds to avoid the problem if
  you wish to delete both files "at the same time":

  1) Ensure the ".conf" file is deleted first.

  2) Delete the ".override" file first, and then wait for a small period
  of time before deleting the corresponding ".conf" file.

  3) Delete the ".override" file first, then call "sudo initctl reload-
  configuration" and then delete the corresponding ".conf" file.

  = Regression Potential =

  None expected. The problem is difficult to trigger anwyay and the
  patch can be seen to correct (what is now) an obvious coding error.

  [1] - http://upstart.ubuntu.com/cookbook/#override-file

  = Original Description =

  ubuntu-touch livefs builds have started killing upstart in the host
  system (in this case, precise, although a similar bug appears to be
  present in current versions).  The livefs build completes, but the
  host dies shortly after launchpad-buildd starts trying to remove the
  build chroot.  The kernel log looks like this:

  Mar 10 13:46:55 allspice kernel: [3743880.621603] init: /home/buildd/build-LIVEFSBUILD-22254/chroot-autobuild/build/chroot/etc/init/tty1.conf: Unable to reload configuration after override deletion
  Mar 10 13:46:55 allspice kernel: [3743880.642455] init: file.c:110: Unhandled error from nih_file_read: No such file or directory
  Mar 10 13:46:55 allspice kernel: [3743880.754281] init: Caught abort, core dumped
  Mar 10 13:46:55 allspice kernel: [3743880.754375] init: file.c:110: Unhandled error from nih_file_read: No such file or directory
  Mar 10 13:46:55 allspice kernel: [3743880.757830] init: Caught abort, core dumped

  This appears to be because a couple of functions call
  conf_reload_path, which may leave an nih_error in place if
  nih_file_read fails, but then do not dispose of the nih_error.  The
  pattern near the end of conf_file_visitor (in precise) is probably
  appropriate.

  We're working around this to some extent in livecd-rootfs by removing
  the .override files first, but it should never be possible for a
  chroot to crash the host's init.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/1430403/+subscriptions