sts-sponsors team mailing list archive

Thread
Date

[Bug 1846787] Re: systemd-logind leaves leftover sessions and scope files

To: sts-sponsors@xxxxxxxxxxxxxxxxxxx
From: Mauricio Faria de Oliveira <mfo@xxxxxxxxxxxxx>
Date: Mon, 02 Dec 2019 23:45:21 -0000
Reply-to: Bug 1846787 <1846787@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx

The autopkgtests regression on docker.io are unrelated to this change.

The failure is on debian/tests/basic-smoke; it happens because
'debootstrap stable <debian>' fails the gpg verification of the Release
file -- the keys used for the Buster stable release are not found in
Xenial's debian-archive-keyring.

This only happens on Xenial; on Bionic and later the debian-archive-
keyring is sufficiently updated.

The workaround is to just patch docker.io/xenial to use 'debootstrap
--no-gpg-check' (the debian debootstrap image is only used to run 'true'
in a container).

The proper solution is a bit more involved on debian-archive-keyring;
discussing this w/ cjwatson.

I'm not sure the proper solution is actually required on this case -- or
even in the general case, as we haven't had bug reports yet about this
since the Buster release in early July.

So we can probably just go with the workaround, but it's a non-runtime
affecting change (build/test-time only), so would have to piggyback on
another SRU to docker.io anyway, i.e., it won't make it to the archive
just to fix this autopkgtest regression.

Thus, we should probably just ignore the docker.io regressions -- I'll
test those w/ PPA build w/ the workaround above.

--
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1846787

Title:
systemd-logind leaves leftover sessions and scope files

Status in dbus package in Ubuntu:
Fix Released
Status in systemd package in Ubuntu:
Fix Released
Status in dbus source package in Xenial:
Fix Committed
Status in systemd source package in Xenial:
Fix Committed

Bug description:
[Impact]
Scope file leakage can cause SSH delays and reduce performance in systemd

[Description]
The current systemd-logind version present in Xenial can leave abandoned SSH
sessions and scope files in cases where the host sees a lot of concurrent SSH
connections. These leftover sessions can slow down systemd performance
greatly, and can have an impact on sshd handling a great number of concurrent
connections.

To fix this issue, patches are needed in both dbus and systemd. These improve the
performance of the communication between dbus and systemd, so that they can
handle a better volume of events (e.g. SSH logins). All of those patches are
already present from Bionic onwards, so we only need those fixes for Xenial.

== Systemd ==
Upstream patches:
- core: use an AF_UNIX/SOCK_DGRAM socket for cgroup agent notification (d8fdc62037b5)

$ git describe --contains d8fdc62037b5
v230~71^2~2

== DBus ==
Upstream patches:
- Only read one message at a time if there are fds pending (892f084eeda0)
- bus: Fix timeout restarts (529600397bca)
- DBusMainLoop: ensure all required timeouts are restarted (446b0d9ac75a)

$ git describe --contains 892f084eeda0 529600397bca 446b0d9ac75a
dbus-1.11.10~44
dbus-1.11.10~45
dbus-1.11.16~2

[Test Case]
1) Simulate a lot of concurrent SSH connections with e.g. a for loop:
multipass@xenial-logind:~$ for i in {1..1000}; do sleep 0.1; ssh localhost sleep 1 & done

2) Check for leaked sessions in /run/systemd/system/:
multipass@xenial-logind:~$ ls -ld /run/systemd/system/session-*.scope*
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-103.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-104.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-105.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-106.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-110.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-111.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-112.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-113.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-114.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-115.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-116.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-117.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-118.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-119.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-120.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-121.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-122.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-123.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-126.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-131.scope.d
drwxr-xr-x 2 root root 160 Oct 4 15:34 /run/systemd/system/session-134.scope.d
...

[Regression Potential]
As the patches change the communication socket between dbus and systemd, possible regressions could cause systemd to not be notified of dbus events and vice-versa. We could see units not getting started properly, and communication between different services break down (e.g. between systemd-logind and other processes).

In this case, the regression potential should be low as these patches
have seen extensive testing both upstream and in more recent releases
of Ubuntu. Nonetheless, these new packages will be rigorously tested
through autopkgtest to avoid any possible Xenial-specific regressions.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/dbus/+bug/1846787/+subscriptions