canonical-ubuntu-qa team mailing list archive

Thread
Date
[Bug 2078790] Re: jammy qemu x86 int3: 0000 [#1] SMP NOPTI

To: canonical-ubuntu-qa@xxxxxxxxxxxxxxxxxxx
From: Mehmet Basaran <2078790@xxxxxxxxxxxxxxxxxx>
Date: Tue, 03 Sep 2024 09:20:34 -0000
Reply-to: Bug 2078790 <2078790@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
jammy:linux-lowlatency-hwe-6.8 6.8.0-44.44.1~22.04.1 qemu-x86 QEMU
Standard PC (i440FX + PIIX, 1996) -> ADT:systemd:upstream-2

5373s --x-- Running TEST-36-NUMAPOLICY --x--
5373s + make -C TEST-36-NUMAPOLICY setup run
5373s make: Entering directory '/tmp/autopkgtest.pW1AKT/build.qvP/src/test/TEST-36-NUMAPOLICY'
5373s Specify build directory with $BUILD_DIR
5373s TEST-36-NUMAPOLICY SETUP: test NUMAPolicy= and NUMAMask= options
5373s Reusing existing cached image /tmp/autopkgtest.pW1AKT/build.qvP/src/test/TEST-36-NUMAPOLICY/../default.img → /tmp/autopkgtest.pW1AKT/build.qvP/src/test/default.img
5373s '/var/tmp/systemd-test.Wk9jIZ/default.img' -> '/tmp/autopkgtest.pW1AKT/build.qvP/src/test/default.img'
5373s I: Masking supporting services
5373s '/var/tmp/systemd-test.Wk9jIZ/root/etc/systemd/system/systemd-hwdb-update.service' -> '/dev/null'
5373s '/var/tmp/systemd-test.Wk9jIZ/root/etc/systemd/system/systemd-journal-catalog-update.service' -> '/dev/null'
5373s '/var/tmp/systemd-test.Wk9jIZ/root/etc/systemd/system/systemd-networkd.service' -> '/dev/null'
5373s '/var/tmp/systemd-test.Wk9jIZ/root/etc/systemd/system/systemd-networkd.socket' -> '/dev/null'
5373s '/var/tmp/systemd-test.Wk9jIZ/root/etc/systemd/system/systemd-resolved.service' -> '/dev/null'
5373s Specify build directory with $BUILD_DIR
5373s TEST-36-NUMAPOLICY RUN: test NUMAPolicy= and NUMAMask= options
5374s + /bin/qemu-system-x86_64 -smp 4 -net none -m 512M -nographic -vga none -kernel /boot/vmlinuz-6.8.0-44-lowlatency -drive format=raw,cache=unsafe,file=/var/tmp/systemd-test.Wk9jIZ/default.img -object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0 -initrd /boot/initrd.img-6.8.0-44-lowlatency -append 'root=/dev/sda1 rw raid=noautodetect rd.luks=0 loglevel=2 init=/lib/systemd/systemd console=ttyS0 selinux=0 SYSTEMD_UNIT_PATH=/usr/lib/systemd/tests/testdata/testsuite-36.units:/usr/lib/systemd/tests/testdata/units: systemd.unit=testsuite.target systemd.wants=testsuite-36.service systemd.wants=end.service'
5374s c[?7l[2J[0mSeaBIOS (version 1.15.0-1)
5384s Booting from ROM..c[?7l[2JLoading, please wait...
5384s Starting version 249.11-0ubuntu3.12
5386s Begin: Loading essential drivers ... done.
5386s Begin: Running /scripts/init-premount ... done.
5386s Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
5387s Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems
5387s done.
5387s Begin: Will now check root file system ... fsck from util-linux 2.37.2
5388s [/usr/sbin/fsck.ext4 (1) -- /dev/sda1] fsck.ext4 -a -C0 /dev/sda1
5388s systemd: clean, 2420/115200 files, 46512/115200 blocks
5388s done.
5388s done.
5388s Begin: Running /scripts/local-bottom ... done.
5388s Begin: Running /scripts/init-bottom ... done.
5389s
5389s Welcome to [1mUbuntu 22.04.4 LTS[0m!
5389s
5390s [[0;32m  OK  [0m] Created slice [0;1;39mSlice /system/modprobe[0m.
5390s [[0;32m  OK  [0m] Created slice [0;1;39mSlice /system/serial-getty[0m.
5390s [[0;32m  OK  [0m] Created slice [0;1;39mUser and Session Slice[0m.
5390s [[0;32m  OK  [0m] Started [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
5390s [[0;32m  OK  [0m] Started [0;1;39mForward Password R…uests to Wall Directory Watch[0m.
5390s [[0;1;38;5;185mUNSUPP[0m] Starting of [0;1;39mArbitrary Exec…m Automount Point[0m unsupported.
5390s [[0;32m  OK  [0m] Reached target [0;1;39mLocal Encrypted Volumes[0m.
5390s [[0;32m  OK  [0m] Reached target [0;1;39mPath Units[0m.
5390s [[0;32m  OK  [0m] Reached target [0;1;39mSlice Units[0m.
5390s [[0;32m  OK  [0m] Reached target [0;1;39mSwaps[0m.
5390s [[0;32m  OK  [0m] Reached target [0;1;39mLocal Verity Protected Volumes[0m.
5390s [[0;32m  OK  [0m] Listening on [0;1;39mProcess Core Dump Socket[0m.
5390s [[0;32m  OK  [0m] Listening on [0;1;39minitctl Compatibility Named Pipe[0m.
5390s [[0;32m  OK  [0m] Listening on [0;1;39mJournal Audit Socket[0m.
5390s [[0;32m  OK  [0m] Listening on [0;1;39mJournal Socket (/dev/log)[0m.
5390s [[0;32m  OK  [0m] Listening on [0;1;39mJournal Socket[0m.
5390s [[0;32m  OK  [0m] Listening on [0;1;39mudev Control Socket[0m.
5390s [[0;32m  OK  [0m] Listening on [0;1;39mudev Kernel Socket[0m.
5390s [[0;32m  OK  [0m] Reached target [0;1;39mSockets[0m.
5390s          Mounting [0;1;39mHuge Pages File System[0m...
5390s          Mounting [0;1;39mPOSIX Message Queue File System[0m...
5390s          Mounting [0;1;39mKernel Debug File System[0m...
5390s          Mounting [0;1;39mKernel Trace File System[0m...
5390s          Starting [0;1;39mJournal Service[0m...
5391s          Starting [0;1;39mLoad Kernel Module configfs[0m...
5391s          Starting [0;1;39mLoad Kernel Module drm[0m...
5391s          Starting [0;1;39mLoad Kernel Module fuse[0m...
5391s          Starting [0;1;39mLoad Kernel Modules[0m...
5391s          Starting [0;1;39mRemount Root and Kernel File Systems[0m...
5391s          Starting [0;1;39mColdplug All udev Devices[0m...
5391s [[0;32m  OK  [0m] Mounted [0;1;39mHuge Pages File System[0m.
5391s [[0;32m  OK  [0m] Mounted [0;1;39mPOSIX Message Queue File System[0m.
5391s [[0;32m  OK  [0m] Mounted [0;1;39mKernel Debug File System[0m.
5391s [[0;32m  OK  [0m] Mounted [0;1;39mKernel Trace File System[0m.
5391s [[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Module configfs[0m.
5391s [[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Module drm[0m.
5391s [[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Module fuse[0m.
5391s          Mounting [0;1;39mFUSE Control File System[0m...
5391s          Mounting [0;1;39mKernel Configuration File System[0m...
5391s [[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Modules[0m.
5391s [[0;32m  OK  [0m] Finished [0;1;39mRemount Root and Kernel File Systems[0m.
5391s          Starting [0;1;39mLoad/Save Random Seed[0m...
5391s          Starting [0;1;39mApply Kernel Variables[0m...
5391s          Starting [0;1;39mCreate System Users[0m...
5391s [[0;32m  OK  [0m] Mounted [0;1;39mFUSE Control File System[0m.
5391s [[0;32m  OK  [0m] Mounted [0;1;39mKernel Configuration File System[0m.
5391s [[0;32m  OK  [0m] Finished [0;1;39mLoad/Save Random Seed[0m.
5391s [[0;32m  OK  [0m] Started [0;1;39mJournal Service[0m.
5391s          Starting [0;1;39mFlush Journal to Persistent Storage[0m...
5391s [[0;32m  OK  [0m] Finished [0;1;39mApply Kernel Variables[0m.
5391s [[0;32m  OK  [0m] Finished [0;1;39mCreate System Users[0m.
5391s          Starting [0;1;39mCreate Static Device Nodes in /dev[0m...
5391s [[0;32m  OK  [0m] Finished [0;1;39mFlush Journal to Persistent Storage[0m.
5391s [[0;32m  OK  [0m] Finished [0;1;39mCreate Static Device Nodes in /dev[0m.
5391s [[0;32m  OK  [0m] Reached target [0;1;39mPreparation for Local File Systems[0m.
5391s [[0;32m  OK  [0m] Reached target [0;1;39mLocal File Systems[0m.
5391s          Starting [0;1;39mCreate Volatile Files and Directories[0m...
5391s          Starting [0;1;39mRule-based Manage…for Device Events and Files[0m...
5391s [[0;32m  OK  [0m] Finished [0;1;39mColdplug All udev Devices[0m.
5392s [[0;32m  OK  [0m] Finished [0;1;39mCreate Volatile Files and Directories[0m.
5392s          Starting [0;1;39mRecord System Boot/Shutdown in UTMP[0m...
5392s [[0;32m  OK  [0m] Finished [0;1;39mRecord System Boot/Shutdown in UTMP[0m.
5392s [[0;32m  OK  [0m] Started [0;1;39mRule-based Manager for Device Events and Files[0m.
5392s [[0;32m  OK  [0m] Reached target [0;1;39mSystem Initialization[0m.
5392s [[0;32m  OK  [0m] Started [0;1;39mDaily Cleanup of Temporary Directories[0m.
5392s [[0;32m  OK  [0m] Reached target [0;1;39mBasic System[0m.
5392s [[0;32m  OK  [0m] Reached target [0;1;39mTimers[0m.
5392s [[0;32m  OK  [0m] Listening on [0;1;39mD-Bus System Message Bus Socket[0m.
5392s          Starting [0;1;39m/etc/rc.local Compatibility[0m...
5392s          Starting [0;1;39mUser Login Management[0m...
5392s [   15.916889] rc.local[258]: /etc/rc.local: 2: iptables: not found
5392s [   15.907320] traps: PANIC: double fault, error_code: 0x0
5392s [   15.909893] double fault: 0000 [#1] PREEMPT SMP NOPTI
5392s [   15.909893] CPU: 3 PID: 252 Comm: systemd-udevd Not tainted 6.8.0-44-lowlatency #44.1~22.04.1-Ubuntu
5392s [   15.909893] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
5392s [   15.911200] RIP: 0010:kmem_cache_alloc+0x93/0x370
5392s [   15.911200] Code: 52 02 00 00 41 f6 44 24 0b 04 0f 85 46 02 00 00 45 31 d2 4d 85 e4 0f 84 33 02 00 00 0f 1f 44 00 00 48 c7 44 24 10 00 00 00 00 <49> 8b 04 24 65 48 03 05 d1 e0 7b 52 48 8b 50 08 48 83 78 10 00 48
5392s [   15.911200] RSP: 0018:ffffa5d6c03dfbf0 EFLAGS: 4376b202
5392s [   15.911200] RAX: ffff95764376b000 RBX: ffff95764376c000 RCX: 0000000005486003
5392s [   15.911200] RDX: 0000000005540003 RSI: 000000000003cf80 RDI: ffff95764376b000
5392s [   15.911200] RBP: ffffa5d6c03dfc40 R08: 0000000000000000 R09: 0000000000000000
5392s [   15.911200] R10: 0000000000000000 R11: 0000000000000000 R12: ffff95764129f600
5392s [   15.911200] R13: 0000000000000cc0 R14: 0000000000001000 R15: ffffffffad8f1b10
5392s [   15.911200] FS:  000071234826f8c0(0000) GS:ffff95765b180000(0000) knlGS:0000000000000000
5392s [   15.911200] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
5392s [   15.911200] CR2: 000076d1398389c5 CR3: 00000000023c4000 CR4: 00000000000006f0
5392s [   15.911200] Call Trace:
5392s [   15.911200]  <#DF>
5392s [   15.911200]  ? show_regs+0x6d/0x80
5392s [   15.911200]  ? die+0x37/0xa0
5392s [   15.911200]  ? exc_double_fault+0xf2/0x1c0
5392s [   15.911200]  ? asm_exc_double_fault+0x1f/0x30
5392s [   15.911200]  ? getname_flags.part.0+0x30/0x1e0
5392s [   15.911200]  ? kmem_cache_alloc+0x93/0x370
5392s [   15.911200]  </#DF>
5392s [   15.911200]  <TASK>
5392s [   15.911200]  ? do_syscall_64+0x8d/0x170
5392s [   15.911200]  getname_flags.part.0+0x30/0x1e0
5392s [   15.911200]  getname_flags+0x40/0x70
5392s [   15.911200]  user_path_at_empty+0x27/0x70
5392s [   15.911200]  do_faccessat+0x10a/0x2f0
5392s [   15.911200]  __x64_sys_access+0x1c/0x30
5392s [   15.911200]  x64_sys_call+0xa08/0x24b0
5392s [   15.911200]  do_syscall_64+0x81/0x170
5392s [   15.911200]  ? scheduler_tick+0x14e/0x370
5392s [   15.911200]  ? invoke_rcu_core+0x51/0x70
5392s [   15.911200]  ? irqentry_exit+0x20/0x50
5392s [   15.915119]  ? common_interrupt+0x54/0xb0
5392s [   15.915119]  ? asm_common_interrupt+0x27/0x40
5392s [   15.915119]  ? note_gp_changes+0x8f/0xa0
5392s [   15.915119]  ? rcu_core+0x1d2/0x390
5392s [   15.915119]  ? rcu_core_si+0xe/0x20
5392s [   15.915119]  ? handle_softirqs+0xdb/0x340
5392s [   15.915119]  ? irqentry_exit_to_user_mode+0x7e/0x260
5392s [   15.915119]  ? irqentry_exit+0x43/0x50
5392s [   15.915119]  entry_SYSCALL_64_after_hwframe+0x78/0x80
5392s [   15.915119] RIP: 0033:0x71234811494b
5392s [   15.915119] Code: 77 05 c3 0f 1f 40 00 48 8b 15 e1 54 10 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff c3 0f 1f 40 00 f3 0f 1e fa b8 15 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 05 c3 0f 1f 40 00 48 8b 15 b1 54 10 00 f7 d8
5392s [   15.915119] RSP: 002b:00007fffb16c6918 EFLAGS: 00000246 ORIG_RAX: 0000000000000015
5392s [   15.915119] RAX: ffffffffffffffda RBX: 00007fffb16c6920 RCX: 000071234811494b
5392s [   15.915119] RDX: 00007fffb16c6937 RSI: 0000000000000000 RDI: 00007fffb16c6920
5392s [   15.915119] RBP: 00007fffb16c69b0 R08: fefefefefefefeff R09: feff5ae353ff6368
5392s [   15.915119] R10: 00007123480b26e0 R11: 0000000000000246 R12: 00007fffb16c6970
5392s [   15.915119] R13: 00007fffb16c6970 R14: 00007fffb16c6970 R15: 00007fffb16c69c0
5392s [   15.915119]  </TASK>
5392s [   15.915119] Modules linked in: btrfs blake2b_generic xor raid6_pq libcrc32c psmouse i2c_piix4 floppy pata_acpi
5392s [   15.915119] ---[ end trace 0000000000000000 ]---
5392s [   15.915119] RIP: 0010:kmem_cache_alloc+0x93/0x370
5392s [   15.915119] Code: 52 02 00 00 41 f6 44 24 0b 04 0f 85 46 02 00 00 45 31 d2 4d 85 e4 0f 84 33 02 00 00 0f 1f 44 00 00 48 c7 44 24 10 00 00 00 00 <49> 8b 04 24 65 48 03 05 d1 e0 7b 52 48 8b 50 08 48 83 78 10 00 48
5392s [   15.915119] RSP: 0018:ffffa5d6c03dfbf0 EFLAGS: 4376b202
5392s [   15.915119] RAX: ffff95764376b000 RBX: ffff95764376c000 RCX: 0000000005486003
5392s [   15.915119] RDX: 0000000005540003 RSI: 000000000003cf80 RDI: ffff95764376b000
5392s [   15.915119] RBP: ffffa5d6c03dfc40 R08: 0000000000000000 R09: 0000000000000000
5392s [   15.915119] R10: 0000000000000000 R11: 0000000000000000 R12: ffff95764129f600
5392s [   15.915119] R13: 0000000000000cc0 R14: 0000000000001000 R15: ffffffffad8f1b10
5392s [   15.915119] FS:  000071234826f8c0(0000) GS:ffff95765b180000(0000) knlGS:0000000000000000
5392s [   15.915119] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
5392s [   15.915119] CR2: 000076d1398389c5 CR3: 00000000023c4000 CR4: 00000000000006f0
5392s [   15.915119] Kernel panic - not syncing: Fatal exception in interrupt
5392s [   15.915119] Kernel Offset: 0x2c400000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
5392s [   15.915119] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
25295s autopkgtest [03:19:35]: ERROR: timed out on command "su -s /bin/bash root -c set -e; exec /tmp/autopkgtest.pW1AKT/wrapper.sh --artifacts=/tmp/autopkgtest.pW1AKT/upstream-2-artifacts --chdir=/tmp/autopkgtest.pW1AKT/build.qvP/src --env=AUTOPKGTEST_TESTBED_ARCH=amd64 --env=AUTOPKGTEST_TEST_ARCH=amd64 --env=DEB_BUILD_OPTIONS=parallel=4 --env=DEBIAN_FRONTEND=noninteractive --env=LANG=C.UTF-8 --unset-env=LANGUAGE --unset-env=LC_ADDRESS --unset-env=LC_ALL --unset-env=LC_COLLATE --unset-env=LC_CTYPE --unset-env=LC_IDENTIFICATION --unset-env=LC_MEASUREMENT --unset-env=LC_MESSAGES --unset-env=LC_MONETARY --unset-env=LC_NAME --unset-env=LC_NUMERIC --unset-env=LC_PAPER --unset-env=LC_TELEPHONE --unset-env=LC_TIME --script-pid-file=/tmp/autopkgtest_script_pid --source-profile --stderr=/tmp/autopkgtest.pW1AKT/upstream-2-stderr --stdout=/tmp/autopkgtest.pW1AKT/upstream-2-stdout --tmp=/tmp/autopkgtest.pW1AKT/autopkgtest_tmp --env=AUTOPKGTEST_NORMAL_USER=ubuntu --env=ADT_NORMAL_USER=ubuntu '--env=ADT_TEST_TRIGGERS=linux-meta-lowlatency-hwe-6.8/6.8.0-44.44.1~22.04.1 linux-lowlatency-hwe-6.8/6.8.0-44.44.1~22.04.1 linux-signed-lowlatency-hwe-6.8/6.8.0-44.44.1~22.04.1' --make-executable=/tmp/autopkgtest.pW1AKT/build.qvP/src/debian/tests/upstream-2 -- /tmp/autopkgtest.pW1AKT/build.qvP/src/debian/tests/upstream-2" (kind: test)
25295s autopkgtest [03:19:35]: test upstream-2: -----------------------]
25295s autopkgtest [03:19:35]: test upstream-2:  - - - - - - - - - - results - - - - - - - - - -
25295s upstream-2           FAIL timed out
25296s autopkgtest [03:19:36]: test boot-smoke: preparing testbed
25348s autopkgtest [03:20:28]: testbed dpkg architecture: amd64
25348s autopkgtest [03:20:28]: testbed apt version: 2.4.12
25348s autopkgtest [03:20:28]: @@@@@@@@@@@@@@@@@@@@ test bed setup

-- 
You received this bug notification because you are a member of Canonical
Platform QA Team, which is subscribed to ubuntu-kernel-tests.
https://bugs.launchpad.net/bugs/2078790

Title:
  jammy qemu x86 int3: 0000 [#1] SMP NOPTI

Status in linux-lowlatency-hwe-6.8 package in Ubuntu:
  New
Status in qemu package in Ubuntu:
  New

Bug description:
  jammy:linux-lowlatency-hwe-6.8 6.8.0-44.44.1~22.04.1 qemu-x86 QEMU
  Standard PC (i440FX + PIIX, 1996)

  
  Recently (2024.08.05), I have been seeing this issue with ADT:systemd:upstream-1/2 test in which kernel panics/prints a stack. I have seen this with jammy:linux-lowlatency-hwe-6.8 and jammy:linux-ibm-6.8. Stack trace is different everytime because kernel receives an interrupt, drop what it is doing, and crash when handling the interrupt.

  I think this is an issue with qemu and not kernel. For jammy, we are
  using qemu 6.2 and there are some fixes related to x86 interrupt
  handling in 8.x
  (https://lore.kernel.org/lkml/20230705215008.GD17440@xxxxxxxxxx/T/). I
  propose we create a launchpad bug and trace the issue. If I am
  correct, we shouldn't see this in noble. And we should occasionally
  see this in 5.15 jammy kernels (and more frequently with lowlantecy
  kernels).

  
  Meanwhile see comments below for some stack traces;

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-lowlatency-hwe-6.8/+bug/2078790/+subscriptions
References

[Bug 2078790] [NEW] jammy qemu x86 int3: 0000 [#1] SMP NOPTI
From: Mehmet Basaran, 2024-09-03