kernel-packages team mailing list archive

[Stéphane Graber <stgraber@xxxxxxxxxxxx>]

Ok, so investigation shows that:

 - LXD bind-mounts all that stuff, it doesn't have a choice as it's not privileged enough to mount things itself
 - mountall fails to run if its "optional" filesystems fail to mount (because that makes a lot of sense...)
 - systemd sets up the host filesystems, on a clean boot they all seem fine
 - "something" apparently remounts debugfs ro sometimes, this breaks containers
 - "something" apparently makes the /proc/sys/fs/binfmt_misc autofs go nuts (loop of symlinks) which also breaks containers

We could try to teach mountall to do the right thing with optional mount
and ignore their failures, however we'd need to SRU that to trusty and
precise and then nag other distros in doing the same (centos, oracle,
rhel, ...) before we can get rid of our workaround.

As a clean Xenial system does work properly, I think it would be best to
figure out what's messing with debugfs and binfmt_misc post-boot and fix
whatever it is to stop doing that.

Would be useful if the bug reporters could document exactly what they
did on their system between the time it worked fine and the time it
stopped working so we can figure out what's messing with those mounts.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1551854

Title:
  LXD bootstrap issues on xenial

Status in linux package in Ubuntu:
  Incomplete

Bug description:
  I'm using lxd with zfs block storage on xenial, and having issues with
  trusty containers. I've witnessed this problem when trying to
  bootstrap as well as after rebooting the host and a container failed
  to start.

  In the latter case, the container that failed to start was the juju
  controller:

  c@mawhrin-skel:~/omnibus-layers$ lxc list                                                                                                        [3/3]
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  |                        NAME                         |  STATE  |              IPV4              | IPV6 |    TYPE    | SNAPSHOTS |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  | juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-0 | STOPPED |                                |      | PERSISTENT |         0 |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  | juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-1 | RUNNING | 10.0.3.28 (eth0)               |      | PERSISTENT |         0 |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  | juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-2 | RUNNING | 10.0.3.85 (eth0)               |      | PERSISTENT |         0 |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  | juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-3 | RUNNING | 10.0.3.176 (eth0)              |      | PERSISTENT |         0 |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  | juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-4 | RUNNING | 10.0.3.66 (eth0)               |      | PERSISTENT |         0 |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  | juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-5 | RUNNING | 10.0.3.31 (eth0)               |      | PERSISTENT |         0 |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  | juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-6 | RUNNING | 10.0.3.196 (eth0)              |      | PERSISTENT |         0 |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+
  | juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-7 | RUNNING | 10.0.3.186 (eth0)              |      | PERSISTENT |         0 |
  |                                                     |         | 10.0.4.1 (lxcbr0)              |      |            |           |
  +-----------------------------------------------------+---------+--------------------------------+------+------------+-----------+

  I manually started it, but found that no upstart services were
  started. Remembering this thread,
  https://lists.ubuntu.com/archives/juju/2016-February/006698.html, I
  checked /var/log/mountall.log in the machine-0 container, and sure
  enough:

  root@juju-5f4bd172-ad22-4726-8d84-47185ab31b54-machine-0:~# cat /var/log/upstart/mountall.log
  mount: permission denied
  mountall: mount /sys/kernel/debug [187] terminated with status 32
  mountall: Filesystem could not be mounted: /sys/kernel/debug

  The problem persists if I stop and start the container. If I remount
  /sys/kernel/debug on the host, then stop and start the container,
  upstart succeeds and the juju controller starts up.

  ProblemType: Bug
  DistroRelease: Ubuntu 16.04
  Package: linux-image-4.4.0-8-generic 4.4.0-8.23
  ProcVersionSignature: Ubuntu 4.4.0-8.23-generic 4.4.2
  Uname: Linux 4.4.0-8-generic x86_64
  NonfreeKernelModules: zfs zunicode zcommon znvpair zavl
  ApportVersion: 2.20-0ubuntu3
  Architecture: amd64
  AudioDevicesInUse:
   USER        PID ACCESS COMMAND
   /dev/snd/controlC0:  c         24562 F.... pulseaudio
  CurrentDesktop: MATE
  Date: Tue Mar  1 10:52:45 2016
  EcryptfsInUse: Yes
  HibernationDevice: RESUME=UUID=007cedda-f922-4e4c-89b1-57b31f18292e
  InstallationDate: Installed on 2016-02-28 (2 days ago)
  InstallationMedia: Ubuntu-MATE 16.04 LTS "Xenial Xerus" - Alpha amd64 (20160224)
  MachineType: LENOVO 2306CTO
  ProcFB: 0 inteldrmfb
  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.4.0-8-generic.efi.signed root=/dev/mapper/ubuntu--mate--vg-root ro quiet splash vt.handoff=7
  RelatedPackageVersions:
   linux-restricted-modules-4.4.0-8-generic N/A
   linux-backports-modules-4.4.0-8-generic  N/A
   linux-firmware                           1.156
  RfKill:
   0: phy0: Wireless LAN
   	Soft blocked: no
   	Hard blocked: no
  SourcePackage: linux
  UpgradeStatus: No upgrade log present (probably fresh install)
  dmi.bios.date: 04/22/2014
  dmi.bios.vendor: LENOVO
  dmi.bios.version: G2ETA1WW (2.61 )
  dmi.board.asset.tag: Not Available
  dmi.board.name: 2306CTO
  dmi.board.vendor: LENOVO
  dmi.board.version: Not Defined
  dmi.chassis.asset.tag: No Asset Information
  dmi.chassis.type: 10
  dmi.chassis.vendor: LENOVO
  dmi.chassis.version: Not Available
  dmi.modalias: dmi:bvnLENOVO:bvrG2ETA1WW(2.61):bd04/22/2014:svnLENOVO:pn2306CTO:pvrThinkPadX230:rvnLENOVO:rn2306CTO:rvrNotDefined:cvnLENOVO:ct10:cvrNotAvailable:
  dmi.product.name: 2306CTO
  dmi.product.version: ThinkPad X230
  dmi.sys.vendor: LENOVO

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1551854/+subscriptions