canonical-ubuntu-qa team mailing list archive

[Jacob Martin <2085470@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

We're seeing test failures caused by the rc-local.service job failing to
start during systemd ADT testing. This occurs on systemd tests
"upstream", "upstream-1", and "upstream-2" depending on the series.

Focal (upstream failing):
...
2159s rc-local.service loaded failed failed /etc/rc.local Compatibility
2159s -rw-r-----+ 1 root systemd-journal 5496832 Oct 18 09:20 /var/tmp/systemd-test.mXcrvC/journal/83ec9cfce1c54880891ba3216d3fb90d/system.journal
2159s -rw-r-----+ 1 root systemd-journal 5496832 Oct 18 09:20 /var/tmp/systemd-test.mXcrvC/journal/83ec9cfce1c54880891ba3216d3fb90d/system@75fc54e19c154ba88471a7a87fc9c9e6-0000000000000001-000624bcd0ed941d.journal
2159s make: *** [Makefile:4: run] Error 1
2159s make: Leaving directory '/tmp/autopkgtest.Bw3WEy/build.stg/src/test/TEST-01-BASIC'
2159s ---- /var/tmp/systemd-test.mXcrvC/journal/83ec9cfce1c54880891ba3216d3fb90d ----
2159s -- Logs begin at Fri 2024-10-18 09:20:37 UTC, end at Fri 2024-10-18 09:20:47 UTC. --
...
2159s Oct 18 09:20:41 systemd-testsuite systemd[1]: rc-local.service: Failed with result 'exit-code'.
2159s Oct 18 09:20:41 systemd-testsuite systemd[1]: Failed to start /etc/rc.local Compatibility.
2159s 
2159s make: Entering directory '/tmp/autopkgtest.Bw3WEy/build.stg/src/test/TEST-01-BASIC'
2159s TEST CLEANUP: Basic systemd setup
2159s make: Leaving directory '/tmp/autopkgtest.Bw3WEy/build.stg/src/test/TEST-01-BASIC'
2159s ========== END: TEST-01-BASIC ==========
...
Full log: https://autopkgtest.ubuntu.com/results/autopkgtest-focal/focal/amd64/s/systemd/20241018_100606_a7e76@/log.gz

Jammy (upstream-1 failing):
...
4033s [   53.216824] systemd[1]: rc-local.service: Failed with result 'exit-code'.
4033s [   53.216961] systemd[1]: Failed to start /etc/rc.local Compatibility.
4033s TEST-01-BASIC: ● rc-local.service loaded failed failed /etc/rc.local Compatibility
4033s -rw-r-----+ 1 root root 8388608 Oct  2 19:10 /tmp/autopkgtest.Hbngvb/upstream-1-artifacts/TEST-01-BASIC.journal
4033s nspawn-root test failed
4033s make: Leaving directory '/tmp/autopkgtest.Hbngvb/build.gbK/src/test/TEST-01-BASIC'
4033s make: *** [Makefile:2: run] Error 1
4033s --x-- Result of TEST-01-BASIC: 2 --x--
...
Full log: https://autopkgtest.ubuntu.com/results/autopkgtest-jammy/jammy/amd64/s/systemd/20241002_193957_25ef9@/log.gz


Noble (upstream-1 and upstream-2 failing):
...
3465s [  124.195642] testsuite-01.sh[86]: + tee /failed
3465s [  124.201747] testsuite-01.sh[86]: ● rc-local.service loaded failed failed /etc/rc.local Compatibility
3465s [  124.202692] testsuite-01.sh[58]: + test '!' -s /failed
3465s [  124.204103] systemd[1]: testsuite-01.service: Failed with result 'exit-code'.
3465s [  124.204425] systemd[1]: Failed to start testsuite-01.service - TEST-01-BASIC.
3465s Spawning getter /bin/journalctl -o export -D /var/tmp/systemd-tests/systemd-test.LksYmA/root/var/log/journal...
3466s Finishing after writing 2582 entries
3466s TEST-01-BASIC: ● rc-local.service loaded failed failed /etc/rc.local Compatibility
3466s -rw-r-----+ 1 root root 8388608 Oct  1 20:50 /tmp/autopkgtest.vltioL/upstream-1-artifacts/TEST-01-BASIC.journal
3466s nspawn-root test failed
3466s make: Leaving directory '/tmp/autopkgtest.vltioL/build.Nhc/src/test/TEST-01-BASIC'
3466s make: *** [Makefile:4: run] Error 1
3466s [20:50:10] --x-- Result of TEST-01-BASIC: 2 --x--
...
Full log: https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/amd64/s/systemd/20241001_223446_455cb@/log.gz

All failures look something like this, where the /etc/rc.local script
appears to fail. All mention not finding an `iptables` binary.

3690s [   20.146013] rc.local[334]: /etc/rc.local: 4: iptables: not found
3690s [   20.148998] rc.local[334]: /etc/rc.local: 8: iptables: not found
3690s [   20.151558] rc.local[334]: /etc/rc.local: 9: iptables: not found
3691s [FAILED] Failed to start rc-local.service.
3691s See 'systemctl status rc-local.service' for details.
3691s [   20.181554] systemd[1]: rc-local.service: Control process exited, code=exited, status=127/n/a
3691s [   20.187259] systemd[1]: rc-local.service: Failed with result 'exit-code'.
3691s [   20.198870] systemd[1]: Failed to start rc-local.service.

The affected systemd tests run in nested VMs that use a root filesystem
generated from components of the host VM's root filesystem that the test
script decides to include. The /etc/rc.local script is included in these
test root filesystems if present, but not iptables binaries.

The upstream-1 case passes when running autopkgtest locally with the
Jammy cloud images (such as https://cloud-
images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img),
seemingly because these images do not include an /etc/rc.local script at
all.

** Affects: autopkgtest (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of
Canonical's Ubuntu QA, which is subscribed to autopkgtest in Ubuntu.
Matching subscriptions: ubuntu-qa-bugs
https://bugs.launchpad.net/bugs/2085470

Title:
  upstream, upstream-1, and upstream-2 systemd tests fail due to failure
  to start rc-local.service

Status in autopkgtest package in Ubuntu:
  New

Bug description:
  We're seeing test failures caused by the rc-local.service job failing
  to start during systemd ADT testing. This occurs on systemd tests
  "upstream", "upstream-1", and "upstream-2" depending on the series.

  Focal (upstream failing):
  ...
  2159s rc-local.service loaded failed failed /etc/rc.local Compatibility
  2159s -rw-r-----+ 1 root systemd-journal 5496832 Oct 18 09:20 /var/tmp/systemd-test.mXcrvC/journal/83ec9cfce1c54880891ba3216d3fb90d/system.journal
  2159s -rw-r-----+ 1 root systemd-journal 5496832 Oct 18 09:20 /var/tmp/systemd-test.mXcrvC/journal/83ec9cfce1c54880891ba3216d3fb90d/system@75fc54e19c154ba88471a7a87fc9c9e6-0000000000000001-000624bcd0ed941d.journal
  2159s make: *** [Makefile:4: run] Error 1
  2159s make: Leaving directory '/tmp/autopkgtest.Bw3WEy/build.stg/src/test/TEST-01-BASIC'
  2159s ---- /var/tmp/systemd-test.mXcrvC/journal/83ec9cfce1c54880891ba3216d3fb90d ----
  2159s -- Logs begin at Fri 2024-10-18 09:20:37 UTC, end at Fri 2024-10-18 09:20:47 UTC. --
  ...
  2159s Oct 18 09:20:41 systemd-testsuite systemd[1]: rc-local.service: Failed with result 'exit-code'.
  2159s Oct 18 09:20:41 systemd-testsuite systemd[1]: Failed to start /etc/rc.local Compatibility.
  2159s 
  2159s make: Entering directory '/tmp/autopkgtest.Bw3WEy/build.stg/src/test/TEST-01-BASIC'
  2159s TEST CLEANUP: Basic systemd setup
  2159s make: Leaving directory '/tmp/autopkgtest.Bw3WEy/build.stg/src/test/TEST-01-BASIC'
  2159s ========== END: TEST-01-BASIC ==========
  ...
  Full log: https://autopkgtest.ubuntu.com/results/autopkgtest-focal/focal/amd64/s/systemd/20241018_100606_a7e76@/log.gz

  Jammy (upstream-1 failing):
  ...
  4033s [   53.216824] systemd[1]: rc-local.service: Failed with result 'exit-code'.
  4033s [   53.216961] systemd[1]: Failed to start /etc/rc.local Compatibility.
  4033s TEST-01-BASIC: ● rc-local.service loaded failed failed /etc/rc.local Compatibility
  4033s -rw-r-----+ 1 root root 8388608 Oct  2 19:10 /tmp/autopkgtest.Hbngvb/upstream-1-artifacts/TEST-01-BASIC.journal
  4033s nspawn-root test failed
  4033s make: Leaving directory '/tmp/autopkgtest.Hbngvb/build.gbK/src/test/TEST-01-BASIC'
  4033s make: *** [Makefile:2: run] Error 1
  4033s --x-- Result of TEST-01-BASIC: 2 --x--
  ...
  Full log: https://autopkgtest.ubuntu.com/results/autopkgtest-jammy/jammy/amd64/s/systemd/20241002_193957_25ef9@/log.gz

  
  Noble (upstream-1 and upstream-2 failing):
  ...
  3465s [  124.195642] testsuite-01.sh[86]: + tee /failed
  3465s [  124.201747] testsuite-01.sh[86]: ● rc-local.service loaded failed failed /etc/rc.local Compatibility
  3465s [  124.202692] testsuite-01.sh[58]: + test '!' -s /failed
  3465s [  124.204103] systemd[1]: testsuite-01.service: Failed with result 'exit-code'.
  3465s [  124.204425] systemd[1]: Failed to start testsuite-01.service - TEST-01-BASIC.
  3465s Spawning getter /bin/journalctl -o export -D /var/tmp/systemd-tests/systemd-test.LksYmA/root/var/log/journal...
  3466s Finishing after writing 2582 entries
  3466s TEST-01-BASIC: ● rc-local.service loaded failed failed /etc/rc.local Compatibility
  3466s -rw-r-----+ 1 root root 8388608 Oct  1 20:50 /tmp/autopkgtest.vltioL/upstream-1-artifacts/TEST-01-BASIC.journal
  3466s nspawn-root test failed
  3466s make: Leaving directory '/tmp/autopkgtest.vltioL/build.Nhc/src/test/TEST-01-BASIC'
  3466s make: *** [Makefile:4: run] Error 1
  3466s [20:50:10] --x-- Result of TEST-01-BASIC: 2 --x--
  ...
  Full log: https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/amd64/s/systemd/20241001_223446_455cb@/log.gz

  All failures look something like this, where the /etc/rc.local script
  appears to fail. All mention not finding an `iptables` binary.

  3690s [   20.146013] rc.local[334]: /etc/rc.local: 4: iptables: not found
  3690s [   20.148998] rc.local[334]: /etc/rc.local: 8: iptables: not found
  3690s [   20.151558] rc.local[334]: /etc/rc.local: 9: iptables: not found
  3691s [FAILED] Failed to start rc-local.service.
  3691s See 'systemctl status rc-local.service' for details.
  3691s [   20.181554] systemd[1]: rc-local.service: Control process exited, code=exited, status=127/n/a
  3691s [   20.187259] systemd[1]: rc-local.service: Failed with result 'exit-code'.
  3691s [   20.198870] systemd[1]: Failed to start rc-local.service.

  The affected systemd tests run in nested VMs that use a root
  filesystem generated from components of the host VM's root filesystem
  that the test script decides to include. The /etc/rc.local script is
  included in these test root filesystems if present, but not iptables
  binaries.

  The upstream-1 case passes when running autopkgtest locally with the
  Jammy cloud images (such as https://cloud-
  images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img),
  seemingly because these images do not include an /etc/rc.local script
  at all.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/autopkgtest/+bug/2085470/+subscriptions