lttng team mailing list archive
-
lttng team
-
Mailing list archive
-
Message #05531
[Bug 1846485] Re: lttng causes crash with 5.0 based kernel
This was reproduced with the linux-generic kernels. The generic kernels
that experience this behavior are: 5.0.0-31-generic and
5.3.0-16-generic.
This behavior was corrected when the following upstream commits were added to the lttng-modules package in Bionic:
2ca0c84f0b ("Fix: mm: create the new vm_fault_t type (v5.1)")
d6cd2c9598 ("Fix: pipe: stop using ->can_merge (v5.1)")
1b7b9c650e ("Fix: Remove start and number from syscall_get_arguments() args (v5.1)")
d88e2fe5c3 ("fix: timer/trace: Improve timer tracing (v5.2)")
36ac1139f0 ("fix: mm/vmscan: drop may_writepage and classzone_idx from direct reclaim begin template (v5.2)")
a6a26911cc ("fix: random: only read from /dev/random after its pool has received 128 bits (v5.2)")
** Package changed: linux-azure (Ubuntu) => lttng-modules (Ubuntu)
--
You received this bug notification because you are a member of Ubuntu
LTTng, which is subscribed to lttng-modules in Ubuntu.
Matching subscriptions: lttng-modules-bugs
https://bugs.launchpad.net/bugs/1846485
Title:
lttng causes crash with 5.0 based kernel
Status in lttng-modules package in Ubuntu:
New
Bug description:
There is a regression with the 5.0 based kernel. The following is the
good and bad kernels:
OS Status Kernel veriosn
18.04.201906170 BAD Linux nt1000000 5.0.0-1014-azure
18.04.201906040 GOOD Linux nt1000000 4.18.0-1020-azure
The crash can be reproduced with this line:
sudo apt-get install lttng-tools lttng-modules-dkms liblttng-ust0
The following trace occurs
2019/09/26 17:36:30.129283 INFO ExtHandler Stopped tracking cgroup: Microsoft.OSTCExtensions.LinuxDiagnostic-2.3.9029, path: /sys/fs/cgroup/cpu/system.slice/Microsoft.OSTCExtensions.LinuxDiagnostic_2.3.9029_fa19de8a-0c72-449e-9754-451dcef4340e.scope
2019/09/26 17:36:30.130284 INFO ExtHandler Stopped tracking cgroup: Microsoft.OSTCExtensions.LinuxDiagnostic-2.3.9029, path: /sys/fs/cgroup/memory/system.slice/Microsoft.OSTCExtensions.LinuxDiagnostic_2.3.9029_fa19de8a-0c72-449e-9754-451dcef4340e.scope
[ 346.729298] PKCS#7 signature not signed with a trusted key
[ 346.735782] PKCS#7 signature not signed with a trusted key
[ 346.744261] PKCS#7 signature not signed with a trusted key
[ 346.750132] PKCS#7 signature not signed with a trusted key
[ 346.755284] PKCS#7 signature not signed with a trusted key
[ 346.768389] PKCS#7 signature not signed with a trusted key
[ 346.778232] BUG: unable to handle kernel paging request at 00000000000032a0
[ 346.778623] #PF error: [normal kernel read fault]
[ 346.778623] PGD 0 P4D 0
[ 346.778623] Oops: 0000 [#1] SMP PTI
[ 346.778623] CPU: 1 PID: 26741 Comm: lttng-sessiond Kdump: loaded Tainted: G OE 5.0.0-1018-azure #19~18.04.1-Ubuntu
[ 346.778623] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090007 06/02/2017
[ 346.778623] RIP: 0010:lttng_tracepoint_notify+0x172/0x210 [lttng_tracer]
[ 346.778623] Code: eb 1a 49 39 c6 0f 85 ab 00 00 00 49 8b 55 10 41 83 c4 01 44 39 a2 8c 02 00 00 76 a3 48 8b 92 90 02 00 00 49 63 c4 4c 63 34 82 <49> 8b 1e 48 89 df e8 33 fbff ff 48 85 c0 49 89 c7 74 52 49 8b 47
[ 346.795750] RSP: 0018:ffff9d5382ecbbd8 EFLAGS: 00010246
[ 346.795750] RAX: 0000000000000000 RBX: ffff9183eb560040 RCX: 0000000000000005
[ 346.795750] RDX: ffffffffc0042420 RSI: 0000000000000001 RDI: ffffffffc0739100
[ 346.795750] RBP: ffff9d5382ecbc08 R08: ffff9183f6c660c0 R09: ffff9183f6807600
[ 346.795750] R10: ffff9d5382ecbb48 R11: 00000000ffffffff R12: 0000000000000000
[ 346.795750] R13: ffff9183eb560040 R14: 00000000000032a0 R15: ffffffffc0739740
[ 346.795750] FS: 00007fd33176c980(0000) GS:ffff9183f6c40000(0000) knlGS:0000000000000000
[ 346.795750] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 346.795750] CR2: 00000000000032a0 CR3: 000000025dec6003 CR4: 00000000001606e0
[ 346.795750] Call Trace:
[ 346.795750] register_tracepoint_module_notifier+0x57/0x80
[ 346.795750] ? 0xffffffffc0778000
[ 346.795750] lttng_tracepoint_init+0x45/0xc40 [lttng_tracer]
[ 346.795750] lttng_events_init+0xbc/0x233 [lttng_tracer]
[ 346.795750] ? 0xffffffffc0778000
[ 346.795750] do_one_initcall+0x4a/0x1be
[ 346.795750] ? _cond_resched+0x19/0x40
[ 346.795750] ? kmem_cache_alloc_trace+0x151/0x1c0
[ 346.795750] do_init_module+0x5f/0x216
[ 346.795750] load_module+0x19f6/0x20a0
[ 346.795750] __do_sys_finit_module+0xfc/0x120
[ 346.795750] ? __do_sys_finit_module+0xfc/0x120
[ 346.795750] __x64_sys_finit_module+0x1a/0x20
[ 346.795750] do_syscall_64+0x61/0x190
[ 346.795750] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 346.795750] RIP: 0033:0x7fd32f9d7839
[ 346.795750] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c348 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
[ 346.795750] RSP: 002b:00007fff8a413f88 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 346.795750] RAX: ffffffffffffffda RBX: 000055e7fc7d1800 RCX: 00007fd32f9d7839
[ 346.795750] RDX: 0000000000000000 RSI: 00007fd331354145 RDI: 0000000000000018
[ 346.795750] RBP: 00007fd331354145 R08: 0000000000000000 R09: 000055e7fc7d5ac0
[ 346.795750] R10: 0000000000000018 R11: 0000000000000246 R12: 0000000000000000
[ 346.795750] R13: 000055e7fc7d6120 R14: 0000000000000000 R15: 000055e7fc7d5ac0
[ 346.795750] Modules linked in: lttng_tracer(OE+) lttng_statedump(OE) lttng_kprobes(OE) lttng_clock(OE) lttng_lib_ring_buffer(OE) lttng_kretprobes(OE) xt_owner xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_security bpfilter nls_iso8859_1 serio_raw hv_balloon joydev sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel hid_generic aesni_intel aes_x86_64 crypto_simd cryptd glue_helper hid_hyperv hyperv_keyboardhid hv_utils hv_netvsc hyperv_fb cfbfillrect cfbimgblt cfbcopyarea pata_acpi
[ 346.795750] CR2: 00000000000032a0
[ 0.000000] Linux version 5.0.0-1018-azure (buildd@lgw01-amd64-048) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #19~18.04.1-Ubuntu SMP Wed Aug 21 05:13:05 UTC 2019 (Ubuntu 5.0.0-1018.19~18.04.1-azure 5.0.21)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.0.0-1018-azure root=UUID=7b4b88a7-6d85-44eb-9155-3132fd3d1cd8 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300 nr_cpus=1 systemd.unit=kdump-tools-dump.service irqpoll nousb ata_piix.prefer_ms_hyperv=0 elfcorehdr=867700K
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Hygon HygonGenuine
[ 0.000000] Centaur CentaurHauls
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000001000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000029000000-0x0000000034f5cfff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000034fffc00-0x0000000034ffffff] usable
[ 0.000000] BIOS-e820: [mem 0x000000003fff0000-0x000000003fffefff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x000000003ffff000-0x000000003fffffff] ACPI NVS
[ 0.000000] printk: bootconsole [earlyser0] enabled
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.3 present.
[ 0.000000] DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090007 06/02/2017
[ 0.000000] Hypervisor detected: Microsoft Hyper-V
[ 0.000000] Hyper-V: features 0x2e7f, hints 0x40c2c
[ 0.000000] Hyper-V Host Build:14393-10.0-0-0.299
[ 0.000000] Hyper-V: LAPIC Timer Frequency: 0xc3500
[ 0.000000] tsc: Marking TSC unstable due to running on Hyper-V
[ 0.000000] Hyper-V: Using hypercall for remote TLB flush
[ 0.000000] tsc: Detected 2394.455 MHz processor
[ 0.001399] last_pfn = 0x35000 max_arch_pfn = 0x400000000
[ 0.004043] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
Memory KASLR using RDRAND RDTSC...
[ 0.024144] found SMP MP-table at [mem 0x000ff780-0x000ff78f]
[ 0.026977] check: Scanning 1 areas for low memory corruption
[ 0.029761] Using GB pages for direct mapping
[ 0.032036] RAMDISK: [mem 0x32625000-0x331fffff]
[ 0.034267] ACPI: Early table checksum verification disabled
[ 0.037090] ACPI: RSDP 0x00000000000F5BF0 000014 (v00 ACPIAM)
[ 0.039763] ACPI: RSDT 0x000000003FFF0000 000040 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.043785] ACPI: FACP 0x000000003FFF0200 000081 (v02 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.048319] ACPI: DSDT 0x000000003FFF1D24 003CBE (v01 MSFTVM MSFTVM02 00000002 INTL 02002026)
[ 0.052474] ACPI: FACS 0x000000003FFFF000 000040
[ 0.054736] ACPI: WAET 0x000000003FFF1A80 000028 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.058789] ACPI: SLIC 0x000000003FFF1AC0 000176 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.062902] ACPI: OEM0 0x000000003FFF1CC0 000064 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.066950] ACPI: SRAT 0x000000003FFF0800 000140 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
[ 0.071109] ACPI: APIC 0x000000003FFF0300 000452 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.075184] ACPI: OEMB 0x000000003FFFF040 000064 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.079460] SRAT: PXM 0 -> APIC 0x00 -> Node 0
[ 0.081643] SRAT: PXM 0 -> APIC 0x01 -> Node 0
[ 0.083847] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x3fffffff] hotplug
[ 0.087160] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0x27fffffff] hotplug
[ 0.090457] ACPI: SRAT: Node 0 PXM 0 [mem 0x280200000-0xfdfffffff] hotplug
[ 0.093834] ACPI: SRAT: Node 0 PXM 0 [mem 0x1000000000-0xffffffffff] hotplug
[ 0.097266] ACPI: SRAT: Node 0 PXM 0 [mem 0x10000200000-0x1ffffffffff] hotplug
[ 0.100827] ACPI: SRAT: Node 0 PXM 0 [mem 0x20000200000-0x3ffffffffff] hotplug
[ 0.109180] NODE_DATA(0) allocated [mem 0x34f32000-0x34f5cfff]
[ 0.112114] Zone ranges:
[ 0.113485] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.116461] DMA32 [mem 0x0000000001000000-0x0000000034ffffff]
[ 0.119510] Normal empty
[ 0.120892] Device empty
[ 0.122274] Movable zone start for each node
[ 0.124384] Early memory node ranges
[ 0.126361] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.129406] node 0: [mem 0x0000000029000000-0x0000000034f5cfff]
[ 0.133029] Zeroed struct page in unavailable ranges: 36869 pages
[ 0.133031] Initmem setup node 0 [mem 0x0000000000001000-0x0000000034f5cfff]
[ 0.143898] ACPI: PM-Timer IO Port: 0x408
[ 0.145927] APIC: NR_CPUS/possible_cpus limit of 1 almost reached. Keeping one slot for boot cpu. Processor 0/0x0 ignored.
[ 0.151414] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.154623] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[ 0.158081] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.161257] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.164541] Using ACPI (MADT) for SMP configuration information
[ 0.167650] smpboot: 128 Processors exceeds NR_CPUS limit of 1
[ 0.170659] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.173280] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.176584] PM: Registered nosave memory: [mem 0x0009f000-0x28ffffff]
[ 0.179894] PM: Registered nosave memory: [mem 0x34f5d000-0x34ffffff]
[ 0.184693] [mem 0x40000000-0xffffffff] available for PCI devices
[ 0.212824] Booting paravirtualized kernel on bare hardware
[ 0.215671] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.221827] random: get_random_bytes called from start_kernel+0x93/0x503 with crng_init=0
[ 0.226026] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
[ 0.230128] percpu: Embedded 45 pages/cpu s147456 r8192 d28672 u2097152
[ 0.233727] Hyper-V: PV spinlocks enabled
[ 0.235483] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes)
[ 0.238597] Built 1 zonelists, mobility grouping on. Total pages: 48357
[ 0.241718] Policy zone: DMA32
[ 0.243155] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.0.0-1018-azure root=UUID=7b4b88a7-6d85-44eb-9155-3132fd3d1cd8 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300 nr_cpus=1 systemd.unit=kdump-tools-dump.service irqpoll nousb ata_piix.prefer_ms_hyperv=0 elfcorehdr=867700K
[ 0.257201] Misrouted IRQ fixup and polling support enabled
[ 0.259980] This may significantly impact system performance
[ 0.272336] Memory: 82316K/196588K available (12292K kernel code, 2232K rwdata, 3840K rodata, 2352K init, 5536K bss, 114272K reserved, 0K cma-reserved)
[ 0.279360] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.282719] Kernel/User page tables isolation: enabled
[ 0.285246] ftrace: allocating 38598 entries in 151 pages
[ 0.303379] rcu: Hierarchical RCU implementation.
[ 0.305670] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=1.
[ 0.308863] Tasks RCU enabled.
[ 0.310334] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.314124] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.320349] NR_IRQS: 524544, nr_irqs: 256, preallocated irqs: 16
[ 0.323901] Spurious LAPIC timer interrupt on cpu 0
[ 0.330364] Console: colour VGA+ 80x25
[ 0.000000] Linux version 5.0.0-1018-azure (buildd@lgw01-amd64-048) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #19~18.04.1-Ubuntu SMP Wed Aug 21 05:13:05 UTC 2019 (Ubuntu 5.0.0-1018.19~18.04.1-azure 5.0.21)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.0.0-1018-azure root=UUID=7b4b88a7-6d85-44eb-9155-3132fd3d1cd8 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300 nr_cpus=1 systemd.unit=kdump-tools-dump.service irqpoll nousb ata_piix.prefer_ms_hyperv=0 elfcorehdr=867700K
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Hygon HygonGenuine
[ 0.000000] Centaur CentaurHauls
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000001000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000029000000-0x0000000034f5cfff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000034fffc00-0x0000000034ffffff] usable
[ 0.000000] BIOS-e820: [mem 0x000000003fff0000-0x000000003fffefff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x000000003ffff000-0x000000003fffffff] ACPI NVS
[ 0.000000] printk: bootconsole [earlyser0] enabled
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.3 present.
[ 0.000000] DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090007 06/02/2017
[ 0.000000] Hypervisor detected: Microsoft Hyper-V
[ 0.000000] Hyper-V: features 0x2e7f, hints 0x40c2c
[ 0.000000] Hyper-V Host Build:14393-10.0-0-0.299
[ 0.000000] Hyper-V: LAPIC Timer Frequency: 0xc3500
[ 0.000000] tsc: Marking TSC unstable due to running on Hyper-V
[ 0.000000] Hyper-V: Using hypercall for remote TLB flush
[ 0.000000] tsc: Detected 2394.455 MHz processor
[ 0.001399] last_pfn = 0x35000 max_arch_pfn = 0x400000000
[ 0.004043] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.024144] found SMP MP-table at [mem 0x000ff780-0x000ff78f]
[ 0.026977] check: Scanning 1 areas for low memory corruption
[ 0.029761] Using GB pages for direct mapping
[ 0.032036] RAMDISK: [mem 0x32625000-0x331fffff]
[ 0.034267] ACPI: Early table checksum verification disabled
[ 0.037090] ACPI: RSDP 0x00000000000F5BF0 000014 (v00 ACPIAM)
[ 0.039763] ACPI: RSDT 0x000000003FFF0000 000040 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.043785] ACPI: FACP 0x000000003FFF0200 000081 (v02 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.048319] ACPI: DSDT 0x000000003FFF1D24 003CBE (v01 MSFTVM MSFTVM02 00000002 INTL 02002026)
[ 0.052474] ACPI: FACS 0x000000003FFFF000 000040
[ 0.054736] ACPI: WAET 0x000000003FFF1A80 000028 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.058789] ACPI: SLIC 0x000000003FFF1AC0 000176 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.062902] ACPI: OEM0 0x000000003FFF1CC0 000064 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.066950] ACPI: SRAT 0x000000003FFF0800 000140 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
[ 0.071109] ACPI: APIC 0x000000003FFF0300 000452 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
[ 0.075184] ACPI: OEMB 0x000000003FFFF040 000064 (v01 VRTUAL MICROSFT 06001702 MSFT 00000097)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lttng-modules/+bug/1846485/+subscriptions
