group.of.nepali.translators team mailing list archive

Thread
Date
[Bug 1870514] Re: [SRU] update containerd:amd64 1.3.3-0 stops docker daemon

To: group.of.nepali.translators@xxxxxxxxxxxxxxxxxxx
From: Launchpad Bug Tracker <1870514@xxxxxxxxxxxxxxxxxx>
Date: Thu, 07 Jan 2021 10:14:41 -0000
Reply-to: Bug 1870514 <1870514@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
This bug was fixed in the package docker.io - 19.03.13-0ubuntu3.1

---------------
docker.io (19.03.13-0ubuntu3.1) groovy; urgency=medium

  * d/p/do_not_bind_docker_to_containerd.patch: Update docker.io to not
    stop when containerd is upgraded, by using Wants= rather than BindTo=.
    (LP: #1870514)
  * d/rules: Fix docker.io to not restart its service during package
    upgrades, to prevent service downtime from automatic updates via
    unattended-upgrade.
    (LP: #1906364)

 -- Bryce Harrington <bryce@xxxxxxxxxxxxx>  Fri, 04 Dec 2020 23:02:49
+0000

** Changed in: docker.io (Ubuntu Groovy)
       Status: Fix Committed => Fix Released

** Changed in: containerd (Ubuntu Xenial)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/1870514

Title:
  [SRU] update containerd:amd64 1.3.3-0 stops docker daemon

Status in containerd package in Ubuntu:
  Fix Released
Status in docker.io package in Ubuntu:
  Fix Released
Status in containerd source package in Xenial:
  Fix Released
Status in docker.io source package in Xenial:
  Fix Released
Status in containerd source package in Bionic:
  Fix Released
Status in docker.io source package in Bionic:
  Fix Released
Status in containerd source package in Focal:
  Fix Released
Status in docker.io source package in Focal:
  Fix Released
Status in containerd source package in Groovy:
  Fix Released
Status in docker.io source package in Groovy:
  Fix Released
Status in containerd source package in Hirsute:
  Fix Released
Status in docker.io source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  Docker uses containerd under the hood.  When containerd is upgraded it
  stops and restarts its service; docker stops when containerd stops but
  doesn’t restart.  Particularly when doing unattended upgrades, an SRU
  fix rolled out for containerd can result in unexpected and widespread
  service outages for docker.

  [Test Case]

  $ sudo apt install docker.io
  $ sudo systemctl start docker
  $ systemctl status docker | grep Active
       Active: active (running) since[...]
  $ systemctl status containerd | grep Active
       Active: active (running) since[...]

  $ docker pull ubuntu/redis:latest
  $ docker run -e REDIS_PASSWORD=1234 --network host \
  --name test-redis -d ubuntu/redis:latest
  $ telnet localhost 6379
  $ docker container logs test-redis

  $ sudo apt install --reinstall containerd
  $ systemctl status containerd | grep Active
       Active: active (running) since
  $ systemctl status docker | grep Active
       Active: inactive (dead) since [...]; 8s ago
  $ docker container logs test-redis

  [Where Problems Could Occur]

  The challenge with this issue is addressing all important corner
  cases, and as such the biggest risk is that we miss a corner case and
  fail to keep the two services running when they should.  Areas to
  watch will be failures during start/stop/restart/upgrade type
  operations.  Issues during runtime are unlikely to relate to this
  change.

  [Original Report]

  hello we have had in several vms, the problem that after updating to "
  containerd:amd64 1.3.3" the docker daemon is stopped and so are all
  running containers

  the vms run with ubuntu 18.04.1/18.04.2

  journal log during the update:

  ```
  -- Logs begin at Tue 2020-01-14 09:58:27 CET, end at Fri 2020-04-03 11:30:39 CEST. --
  Apr 03 06:09:09 server dockerd[1751]: time="2020-04-03T06:09:09+02:00" level=info msg="level=info ts=2020-04-03T04:09:09.648358396Z caller=loki.go:55 container_id=1eefb971e84cdaa4eadb0ac890fe9adf5275a4e9cbcf9635cf941b3aa469dd1f msg=\"ignoring empty line\" line=" plugin=255e05f2d4535991812f30e4cb5ea39fe0b498ca612c26dd89909de318b7f68d
  Apr 03 06:09:09 server dockerd[1751]: time="2020-04-03T06:09:09+02:00" level=info msg="level=info ts=2020-04-03T04:09:09.660781933Z caller=loki.go:55 container_id=1eefb971e84cdaa4eadb0ac890fe9adf5275a4e9cbcf9635cf941b3aa469dd1f msg=\"ignoring empty line\" line=\"    \"" plugin=255e05f2d4535991812f30e4cb5ea39fe0b498ca612c26dd89909de318b7f68d
  Apr 03 06:09:14 server interface_rules[1485]: {"time":"2020-04-03 06:09:14+02:00","message":"interface enp0s3 works fine"}
  Apr 03 06:09:14 server interface_rules[1485]: {"time":"2020-04-03 06:09:14+02:00","message":"interface enp0s8 works fine"}
  Apr 03 06:09:24 server dockerd[1751]: time="2020-04-03T06:09:24+02:00" level=info msg="level=info ts=2020-04-03T04:09:24.650166885Z caller=loki.go:55 container_id=1eefb971e84cdaa4eadb0ac890fe9adf5275a4e9cbcf9635cf941b3aa469dd1f msg=\"ignoring empty line\" line=" plugin=255e05f2d4535991812f30e4cb5ea39fe0b498ca612c26dd89909de318b7f68d
  Apr 03 06:09:24 server dockerd[1751]: time="2020-04-03T06:09:24+02:00" level=info msg="level=info ts=2020-04-03T04:09:24.663239678Z caller=loki.go:55 container_id=1eefb971e84cdaa4eadb0ac890fe9adf5275a4e9cbcf9635cf941b3aa469dd1f msg=\"ignoring empty line\" line=\"    \"" plugin=255e05f2d4535991812f30e4cb5ea39fe0b498ca612c26dd89909de318b7f68d
  Apr 03 06:09:31 server systemd[1]: Starting Daily apt upgrade and clean activities...
  Apr 03 06:09:39 server dockerd[1751]: time="2020-04-03T06:09:39+02:00" level=info msg="level=info ts=2020-04-03T04:09:39.64981528Z caller=loki.go:55 container_id=1eefb971e84cdaa4eadb0ac890fe9adf5275a4e9cbcf9635cf941b3aa469dd1f msg=\"ignoring empty line\" line=" plugin=255e05f2d4535991812f30e4cb5ea39fe0b498ca612c26dd89909de318b7f68d
  Apr 03 06:09:39 server dockerd[1751]: time="2020-04-03T06:09:39+02:00" level=info msg="level=info ts=2020-04-03T04:09:39.665086076Z caller=loki.go:55 container_id=1eefb971e84cdaa4eadb0ac890fe9adf5275a4e9cbcf9635cf941b3aa469dd1f msg=\"ignoring empty line\" line=\"    \"" plugin=255e05f2d4535991812f30e4cb5ea39fe0b498ca612c26dd89909de318b7f68d
  Apr 03 06:09:43 server systemd[1]: Stopping Docker Application Container Engine...
  Apr 03 06:09:43 server dockerd[1751]: time="2020-04-03T06:09:43.571408106+02:00" level=info msg="Processing signal 'terminated'"
  Apr 03 06:09:43 server dockerd[1751]: time="2020-04-03T06:09:43.614841270+02:00" level=info msg="Daemon shutdown complete"
  Apr 03 06:09:43 server dockerd[1751]: time="2020-04-03T06:09:43.614972021+02:00" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=moby
  Apr 03 06:09:43 server systemd[1]: Stopped Docker Application Container Engine.
  Apr 03 06:09:43 server systemd[1]: Stopping containerd container runtime...
  Apr 03 06:09:43 server systemd[1]: Closed Docker Socket for the API.
  Apr 03 06:09:43 server containerd[1736]: time="2020-04-03T06:09:43.896546147+02:00" level=info msg="Stop CRI service"
  Apr 03 06:09:43 server containerd[1736]: time="2020-04-03T06:09:43.897819071+02:00" level=info msg="Stop CRI service"
  Apr 03 06:09:43 server containerd[1736]: time="2020-04-03T06:09:43.899864471+02:00" level=info msg="Event monitor stopped"
  Apr 03 06:09:43 server containerd[1736]: time="2020-04-03T06:09:43.900291027+02:00" level=info msg="Stream server stopped"
  Apr 03 06:09:43 server systemd[1]: Stopped containerd container runtime.
  Apr 03 06:09:47 server systemd[1]: Reloading.
  Apr 03 06:09:48 server systemd[1]: Reloading.
  Apr 03 06:09:49 server systemd[1]: Reloading.
  Apr 03 06:09:49 server systemd[1]: Starting containerd container runtime...
  Apr 03 06:09:49 server containerd[1904]: time="2020-04-03T06:09:49.951538613+02:00" level=info msg="starting containerd" revision= version="1.3.3-0ubuntu1~18.04.2"
  Apr 03 06:09:49 server systemd[1]: Started containerd container runtime.
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.021260455+02:00" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.021461398+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.022876671+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.022992123+02:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.023036103+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.031683852+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.031948994+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.032592571+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.034455570+02:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.034808794+02:00" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.034914325+02:00" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.034952775+02:00" level=info msg="metadata content store policy set" policy=shared
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035299108+02:00" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035362929+02:00" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035585401+02:00" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035642662+02:00" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035676962+02:00" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035751653+02:00" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035797984+02:00" level=info msg="loading plugin \"io.containerd.service.v1.leases-service\"..." type=io.containerd.service.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035899825+02:00" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035955175+02:00" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.035994665+02:00" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.504599571+02:00" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.504847245+02:00" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.507240519+02:00" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.514716375+02:00" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515138699+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515385472+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515439083+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515474183+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515521113+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515569144+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515602784+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515645985+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515812346+02:00" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.515960658+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.516016449+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.516062909+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.516218490+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.cri\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.516437722+02:00" level=info msg="Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc DefaultRuntime:{Type: Engine: PodAnnotations:[] Root: Options:<nil> PrivilegedWithoutHostDevices:false} UntrustedWorkloadRuntime:{Type: Engine: PodAnnotations:[] Root: Options:<nil> PrivilegedWithoutHostDevices:false} Runtimes:map[runc:{Type:io.containerd.runc.v1 Engine: PodAnnotations:[] Root: Options:<nil> PrivilegedWithoutHostDevices:false}] NoPivot:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginConfTemplate:} Registry:{Mirrors:map[docker.io:{Endpoints:[https://registry-1.docker.io]}] Configs:map[] Auths:map[]} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SandboxImage:k8s.gcr.io/pause:3.1 StatsCollectPeriod:10 SystemdCgroup:false EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false} ContainerdRootDir:/var/lib/containerd ContainerdEndpoint:/run/containerd/containerd.sock RootDir:/var/lib/containerd/io.containerd.grpc.v1.cri StateDir:/run/containerd/io.containerd.grpc.v1.cri}"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.516732916+02:00" level=info msg="Connect containerd service"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.517528134+02:00" level=info msg="Get image filesystem path \"/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs\""
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.517888717+02:00" level=error msg="Failed to load cni during init, please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.518436663+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.518837507+02:00" level=info msg="Start subscribing containerd event"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.519057900+02:00" level=info msg="Start recovering state"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.519208531+02:00" level=info msg=serving... address=/run/containerd/containerd.sock.ttrpc
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.519444914+02:00" level=info msg=serving... address=/run/containerd/containerd.sock
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.519481864+02:00" level=info msg="Start event monitor"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.519573005+02:00" level=info msg="Start snapshots syncer"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.519657075+02:00" level=info msg="Start streaming server"
  Apr 03 06:09:50 server containerd[1904]: time="2020-04-03T06:09:50.519497564+02:00" level=info msg="containerd successfully booted in 0.569453s"
  Apr 03 06:10:08 server systemd[1]: Reloading.
  Apr 03 06:10:08 server systemd[1]: Stopping LSB: automatic crash report generation...
  Apr 03 06:10:08 server apport[2377]:  * Stopping automatic crash report generation: apport
  Apr 03 06:10:08 server apport[2377]:    ...done.
  Apr 03 06:10:08 server systemd[1]: Stopped LSB: automatic crash report generation.
  Apr 03 06:10:09 server systemd[1]: Reloading.
  Apr 03 06:10:09 server systemd[1]: Reloading.
  Apr 03 06:10:11 server systemd[1]: Reloading.
  Apr 03 06:10:11 server systemd[1]: Reloading.
  Apr 03 06:10:12 server systemd[1]: Reloading.
  Apr 03 06:10:12 server systemd[1]: Starting LSB: automatic crash report generation...
  Apr 03 06:10:12 server apport[2666]:  * Starting automatic crash report generation: apport
  Apr 03 06:10:12 server apport[2666]:    ...done.
  Apr 03 06:10:12 server systemd[1]: Started LSB: automatic crash report generation.
  Apr 03 06:10:12 server systemd[1]: Reloading.
  Apr 03 06:10:18 server systemd[1]: Started Daily apt upgrade and clean activities.

  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/containerd/+bug/1870514/+subscriptions