kernel-packages team mailing list archive
-
kernel-packages team
-
Mailing list archive
-
Message #33122
[Bug 1251946] Re: Network related kernel panic on Atom 64bit system using saucy backport stack on precise
Ok, one more shot. It looks like PPPOE wasn't enabled for amd64 in
mainline. I enabled it and rebuilt a test kernel.
Again, its up to the following commit:
6be35c700f742e911ecedd07fcc43d4439922334
The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1251946
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1251946
Title:
Network related kernel panic on Atom 64bit system using saucy backport
stack on precise
Status in “linux” package in Ubuntu:
Confirmed
Bug description:
I can't report this through ubuntu-bug as the system is a firewall
with restricted connectivity and the kernel bug prevents me from using
the system for more than a couple minutes anyway.
I recently deployed the saucy kernel on that precise system, after the
first reboot things looked fine but just a few minutes later, the
whole system hang. I reproduced this very reliably 3-4 times, then
hooked up a console cable and extracted the following kernel panic:
[ 124.428031] ------------[ cut here ]------------
[ 124.432656] Kernel BUG at ffffffff81629ce8 [verbose debug info unavailable]
[ 124.439622] invalid opcode: 0000 [#1] SMP
[ 124.443766] Modules linked in: authenc(F) esp6(F) xfrm6_mode_transport(F) ipcomp6(F) xfrm6_tunnel(F) tunnel6(F) xfrm4_mode_tunnel(F) xfrm6_mode_tunnel(F) v)
[ 124.577657] CPU: 0 PID: 12800 Comm: samba Tainted: GF WC 3.11.0-13-generic #20~precise2-Ubuntu
[ 124.586962] Hardware name: /D2500CC, BIOS CCCDT10N.86A.0039.2013.0425.1625 04/25/2013
[ 124.596268] task: ffff8800cd711770 ti: ffff8800bfde2000 task.ti: ffff8800bfde2000
[ 124.603752] RIP: 0010:[<ffffffff81629ce8>] [<ffffffff81629ce8>] pskb_expand_head+0x288/0x2d0
[ 124.612305] RSP: 0018:ffff88012fc03688 EFLAGS: 00010202
[ 124.617621] RAX: 0000000000000003 RBX: ffff8800cd44d000 RCX: 0000000000000020
[ 124.624762] RDX: 0000000000000000 RSI: 0000000000000050 RDI: 00000000000006c0
[ 124.631900] RBP: ffff88012fc036c8 R08: 3f2c680500000060 R09: 3f2c680500000060
[ 124.639037] R10: bfcb2ffeff3e1602 R11: 20270ff0c0f20726 R12: 0000000000000050
[ 124.646175] R13: 0000000000000020 R14: 0000000000000001 R15: ffffffff81709280
[ 124.653317] FS: 00007f81c5448740(0000) GS:ffff88012fc00000(0000) knlGS:0000000000000000
[ 124.661407] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 124.667158] CR2: 000000000330e780 CR3: 00000000bfdc9000 CR4: 00000000000007f0
[ 124.674295] Stack:
[ 124.676318] ffffe8ffffc00640 0000000180400040 ffff8801264d6d40 ffff8800cd44d000
[ 124.683792] ffff880126ddc000 ffff880126919180 0000000000000001 ffffffff81709280
[ 124.691262] ffff88012fc036f8 ffffffff816caac0 ffff8800cd44d000 0000000000000001
[ 124.698729] Call Trace:
[ 124.701187] <IRQ>
[ 124.703121] [<ffffffff81709280>] ? xfrm6_extract_output+0x50/0x50
[ 124.709525] [<ffffffff816caac0>] xfrm_output_one+0x90/0x230
[ 124.715189] [<ffffffff816cad26>] xfrm_output_resume+0xc6/0x170
[ 124.721112] [<ffffffff816cade3>] xfrm_output2+0x13/0x20
[ 124.726429] [<ffffffff816cae2f>] xfrm_output+0x3f/0x100
[ 124.731747] [<ffffffff81709299>] xfrm6_output_finish+0x19/0x20
[ 124.737669] [<ffffffff816d5729>] ip6_fragment+0x929/0xab0
[ 124.743160] [<ffffffff81709280>] ? xfrm6_extract_output+0x50/0x50
[ 124.749344] [<ffffffff81708ed0>] ? xfrm6_local_rxpmtu+0x70/0x70
[ 124.755356] [<ffffffff81708fa2>] __xfrm6_output+0xd2/0x180
[ 124.760933] [<ffffffff817092ca>] xfrm6_output+0x2a/0x70
[ 124.766250] [<ffffffff816d4a03>] ip6_forward+0x243/0x640
[ 124.771654] [<ffffffff816d5a20>] ? ip6_output+0xb0/0xb0
[ 124.776971] [<ffffffff816d5aa8>] ip6_rcv_finish+0x88/0x90
[ 124.782460] [<ffffffff816d5a20>] ? ip6_output+0xb0/0xb0
[ 124.787787] [<ffffffffa027783d>] __ipv6_conntrack_in+0xdd/0x180 [nf_conntrack_ipv6]
[ 124.795539] [<ffffffffa0277905>] ipv6_conntrack_in+0x25/0x30 [nf_conntrack_ipv6]
[ 124.803027] [<ffffffff816660df>] nf_iterate+0x8f/0xd0
[ 124.808172] [<ffffffff816d5a20>] ? ip6_output+0xb0/0xb0
[ 124.813488] [<ffffffff8166619d>] nf_hook_slow+0x7d/0x150
[ 124.818891] [<ffffffff816d5a20>] ? ip6_output+0xb0/0xb0
[ 124.824210] [<ffffffff816d5a20>] ? ip6_output+0xb0/0xb0
[ 124.829532] [<ffffffffa021b3e2>] nf_ct_frag6_output+0xf2/0x100 [nf_defrag_ipv6]
[ 124.836932] [<ffffffff816d5a20>] ? ip6_output+0xb0/0xb0
[ 124.842255] [<ffffffffa021a0a8>] ipv6_defrag.part.2+0x98/0xe0 [nf_defrag_ipv6]
[ 124.849577] [<ffffffffa021a11c>] ipv6_defrag+0x2c/0x30 [nf_defrag_ipv6]
[ 124.856284] [<ffffffff816660df>] nf_iterate+0x8f/0xd0
[ 124.861429] [<ffffffff816d5a20>] ? ip6_output+0xb0/0xb0
[ 124.866743] [<ffffffff8166619d>] nf_hook_slow+0x7d/0x150
[ 124.872146] [<ffffffff816d5a20>] ? ip6_output+0xb0/0xb0
[ 124.877467] [<ffffffff816d6213>] ipv6_rcv+0x313/0x3f0
[ 124.882609] [<ffffffff81636cba>] __netif_receive_skb_core+0x5ca/0x720
[ 124.889140] [<ffffffff81636e31>] __netif_receive_skb+0x21/0x70
[ 124.895063] [<ffffffff81636ea3>] netif_receive_skb+0x23/0x90
[ 124.900825] [<ffffffffa02fe6af>] br_pass_frame_up+0x9f/0x110 [bridge]
[ 124.907364] [<ffffffffa02fe8fb>] br_handle_frame_finish+0x1db/0x330 [bridge]
[ 124.914509] [<ffffffffa02febe4>] br_handle_frame+0x194/0x260 [bridge]
[ 124.921050] [<ffffffffa02fea50>] ? br_handle_frame_finish+0x330/0x330 [bridge]
[ 124.928362] [<ffffffff81636a69>] __netif_receive_skb_core+0x379/0x720
[ 124.934894] [<ffffffff810978e6>] ? ttwu_queue+0xb6/0xd0
[ 124.940210] [<ffffffff81636e31>] __netif_receive_skb+0x21/0x70
[ 124.946135] [<ffffffff816376f1>] process_backlog+0xb1/0x190
[ 124.951799] [<ffffffff81637f34>] net_rx_action+0x134/0x260
[ 124.957378] [<ffffffff8106a510>] __do_softirq+0xe0/0x280
[ 124.962780] [<ffffffff8175071c>] call_softirq+0x1c/0x30
[ 124.968094] <EOI>
[ 124.970029] [<ffffffff81015e35>] do_softirq+0x65/0xa0
[ 124.975391] [<ffffffff8106a1a4>] local_bh_enable+0x94/0xa0
[ 124.980970] [<ffffffff816d2c31>] ip6_finish_output2+0x1e1/0x4b0
[ 124.986979] [<ffffffff816d2a50>] ? ip6_flush_pending_frames+0xb0/0xb0
[ 124.993509] [<ffffffff816d5729>] ip6_fragment+0x929/0xab0
[ 124.999002] [<ffffffff816d2a50>] ? ip6_flush_pending_frames+0xb0/0xb0
[ 125.005532] [<ffffffff816d58b0>] ? ip6_fragment+0xab0/0xab0
[ 125.011196] [<ffffffff816d21f0>] ? ac6_proc_exit+0x20/0x20
[ 125.016774] [<ffffffff816d5931>] ip6_finish_output+0x81/0xc0
[ 125.022523] [<ffffffff816d59ac>] ip6_output+0x3c/0xb0
[ 125.027667] [<ffffffff816d4332>] ? __ip6_local_out+0x72/0x80
[ 125.033418] [<ffffffff816d4369>] ip6_local_out+0x29/0x30
[ 125.038821] [<ffffffff816d4643>] ip6_push_pending_frames+0x2d3/0x450
[ 125.045268] [<ffffffff816ed236>] udp_v6_push_pending_frames+0x136/0x320
[ 125.051972] [<ffffffff816ee228>] udpv6_sendmsg+0x908/0xb30
[ 125.057551] [<ffffffff813319ae>] ? aa_net_perm+0xae/0x110
[ 125.063040] [<ffffffff816da84a>] ? ipv6_dev_get_saddr+0x22a/0x320
[ 125.069225] [<ffffffff816a58d1>] inet_sendmsg+0x61/0xb0
[ 125.074542] [<ffffffff81331a86>] ? aa_revalidate_sk+0x76/0x80
[ 125.080378] [<ffffffff8132c0f7>] ? apparmor_socket_sendmsg+0x17/0x20
[ 125.086823] [<ffffffff8161d322>] sock_sendmsg+0xc2/0xe0
[ 125.092143] [<ffffffff81089aed>] ? add_wait_queue+0x4d/0x60
[ 125.097805] [<ffffffff811fb18c>] ? ep_scan_ready_list.isra.11+0x1bc/0x1c0
[ 125.104683] [<ffffffff81620348>] SYSC_sendto+0x138/0x190
[ 125.110087] [<ffffffff81745ba5>] ? _raw_spin_lock_irq+0x15/0x20
[ 125.116096] [<ffffffff811fbee6>] ? SyS_epoll_wait+0xd6/0xf0
[ 125.121761] [<ffffffff8162097e>] SyS_sendto+0xe/0x10
[ 125.126819] [<ffffffff8174ed1d>] system_call_fastpath+0x1a/0x1f
[ 125.132824] Code: b2 ff e9 16 ff ff ff b8 f4 ff ff ff eb a6 48 89 d7 48 89 55 c0 e8 49 cb b2 ff 84 c0 48 8b 55 c0 0f 85 9c fe ff ff e9 93 fe ff ff <0f> 0b
[ 125.152869] RIP [<ffffffff81629ce8>] pskb_expand_head+0x288/0x2d0
[ 125.159072] RSP <ffff88012fc03688>
[ 125.162682] ---[ end trace 389dbe75bb7ab4d0 ]---
[ 125.167349] Kernel panic - not syncing: Fatal exception in interrupt
Additionally, the following Opps (looks unrelated but we never know) appears at boot time:
Nov 17 02:41:33 sateda kernel: [ 5.767282] gma500 0000:00:02.0: setting latency timer to 64
Nov 17 02:41:33 sateda kernel: [ 5.767457] gma500 0000:00:02.0: irq 47 for MSI/MSI-X
Nov 17 02:41:33 sateda kernel: [ 5.767569] gma500 0000:00:02.0: GPU: power management timed out.
Nov 17 02:41:33 sateda kernel: [ 5.795072] resource map sanity check conflict: 0xcf800000 0xcffbefff 0xcf800000 0xcf9fffff PCI Bus 0000:02
Nov 17 02:41:33 sateda kernel: [ 5.795081] ------------[ cut here ]------------
Nov 17 02:41:33 sateda kernel: [ 5.795094] WARNING: CPU: 0 PID: 517 at /build/buildd/linux-lts-saucy-3.11.0/arch/x86/mm/ioremap.c:171 __ioremap_caller+0x382/0x390()
Nov 17 02:41:33 sateda kernel: [ 5.795098] Info: mapping multiple BARs. Your kernel is fine.
Nov 17 02:41:33 sateda kernel: [ 5.795101] Modules linked in: snd_hda_intel(F+) gma500_gfx(F+) snd_hda_codec(F) nls_iso8859_1(F) snd_hwdep(F) snd_pcm(F) drm_kms_helper(F) xt_tcpudp(F) drm(F) psmouse(F) snd_timer(F) snd(F) serio_raw(F) lpc_ich(F) soundcore(F) i2c_algo_bit(F) snd_page_alloc(F) parport_pc(F) iptable_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) video(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack(F) mac_hid(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) lp(F) parport(F) ip_tables(F) x_tables(F) ahci(F) e1000e(F) ptp(F) pps_core(F) libahci(F)
Nov 17 02:41:33 sateda kernel: [ 5.795165] CPU: 0 PID: 517 Comm: modprobe Tainted: GF 3.11.0-13-generic #20~precise2-Ubuntu
Nov 17 02:41:33 sateda kernel: [ 5.795169] Hardware name: /D2500CC, BIOS CCCDT10N.86A.0039.2013.0425.1625 04/25/2013
Nov 17 02:41:33 sateda kernel: [ 5.795174] 00000000000000ab ffff88012837b958 ffffffff8173a05d 00000000000015d0
Nov 17 02:41:33 sateda kernel: [ 5.795182] ffff88012837b9a8 ffff88012837b998 ffffffff8106539c ffff88012837b9b8
Nov 17 02:41:33 sateda kernel: [ 5.795190] ffffc90000b80000 00000000cf800000 00000000cf800000 00000000007bf000
Nov 17 02:41:33 sateda kernel: [ 5.795197] Call Trace:
Nov 17 02:41:33 sateda kernel: [ 5.795208] [<ffffffff8173a05d>] dump_stack+0x46/0x58
Nov 17 02:41:33 sateda kernel: [ 5.795217] [<ffffffff8106539c>] warn_slowpath_common+0x8c/0xc0
Nov 17 02:41:33 sateda kernel: [ 5.795223] [<ffffffff81065486>] warn_slowpath_fmt+0x46/0x50
Nov 17 02:41:33 sateda kernel: [ 5.795230] [<ffffffff81055f52>] __ioremap_caller+0x382/0x390
Nov 17 02:41:33 sateda kernel: [ 5.795249] [<ffffffffa0223dde>] ? psb_gtt_init+0x47e/0x580 [gma500_gfx]
Nov 17 02:41:33 sateda kernel: [ 5.795257] [<ffffffff8105609e>] ioremap_wc+0x2e/0x30
Nov 17 02:41:33 sateda kernel: [ 5.795272] [<ffffffffa0223dde>] psb_gtt_init+0x47e/0x580 [gma500_gfx]
Nov 17 02:41:33 sateda kernel: [ 5.795288] [<ffffffffa02282b4>] psb_driver_load+0x134/0x360 [gma500_gfx]
Nov 17 02:41:33 sateda kernel: [ 5.795317] [<ffffffffa0173de1>] drm_get_pci_dev+0x181/0x2e0 [drm]
Nov 17 02:41:33 sateda kernel: [ 5.795334] [<ffffffffa0227e65>] psb_probe+0x15/0x20 [gma500_gfx]
Nov 17 02:41:33 sateda kernel: [ 5.795341] [<ffffffff813abe3b>] local_pci_probe+0x4b/0x80
Nov 17 02:41:33 sateda kernel: [ 5.795348] [<ffffffff813ad6b9>] __pci_device_probe+0xd9/0xe0
Nov 17 02:41:33 sateda kernel: [ 5.795355] [<ffffffff813ad6fa>] pci_device_probe+0x3a/0x60
Nov 17 02:41:33 sateda kernel: [ 5.795363] [<ffffffff8149389c>] really_probe+0x6c/0x330
Nov 17 02:41:33 sateda kernel: [ 5.795369] [<ffffffff81493ce7>] driver_probe_device+0x47/0xa0
Nov 17 02:41:33 sateda kernel: [ 5.795376] [<ffffffff81493deb>] __driver_attach+0xab/0xb0
Nov 17 02:41:33 sateda kernel: [ 5.795382] [<ffffffff81493d40>] ? driver_probe_device+0xa0/0xa0
Nov 17 02:41:33 sateda kernel: [ 5.795389] [<ffffffff81491ace>] bus_for_each_dev+0x5e/0x90
Nov 17 02:41:33 sateda kernel: [ 5.795395] [<ffffffff8149345e>] driver_attach+0x1e/0x20
Nov 17 02:41:33 sateda kernel: [ 5.795402] [<ffffffff81492eec>] bus_add_driver+0x10c/0x290
Nov 17 02:41:33 sateda kernel: [ 5.795408] [<ffffffff8149436d>] driver_register+0x7d/0x160
Nov 17 02:41:33 sateda kernel: [ 5.795418] [<ffffffffa0255000>] ? 0xffffffffa0254fff
Nov 17 02:41:33 sateda kernel: [ 5.795425] [<ffffffff813ac67c>] __pci_register_driver+0x4c/0x50
Nov 17 02:41:33 sateda kernel: [ 5.795445] [<ffffffffa017405a>] drm_pci_init+0x11a/0x130 [drm]
Nov 17 02:41:33 sateda kernel: [ 5.795452] [<ffffffffa0255000>] ? 0xffffffffa0254fff
Nov 17 02:41:33 sateda kernel: [ 5.795469] [<ffffffffa0255017>] psb_init+0x17/0x1000 [gma500_gfx]
Nov 17 02:41:33 sateda kernel: [ 5.795476] [<ffffffff8100212a>] do_one_initcall+0xfa/0x1b0
Nov 17 02:41:33 sateda kernel: [ 5.795483] [<ffffffff810578d3>] ? set_memory_nx+0x43/0x50
Nov 17 02:41:33 sateda kernel: [ 5.795493] [<ffffffff8172e724>] do_init_module+0x80/0x1d1
Nov 17 02:41:33 sateda kernel: [ 5.795501] [<ffffffff810d1a49>] load_module+0x4c9/0x5f0
Nov 17 02:41:33 sateda kernel: [ 5.795507] [<ffffffff810cf240>] ? add_kallsyms+0x210/0x210
Nov 17 02:41:33 sateda kernel: [ 5.795514] [<ffffffff810d1c24>] SyS_init_module+0xb4/0x100
Nov 17 02:41:33 sateda kernel: [ 5.795522] [<ffffffff8174ed1d>] system_call_fastpath+0x1a/0x1f
Nov 17 02:41:33 sateda kernel: [ 5.795526] ---[ end trace c25c2c18f7a6e9ab ]---
Nov 17 02:41:33 sateda kernel: [ 5.824962] acpi device:29: registered as cooling_device2
The system itself is a D2500 atom system, running an up to date Ubuntu 12.04 64bit, it has a considerable amount of network interfaces, including bonds, bridges and most of its traffic is IPv6.
None of the kernel traces above appear when running the original 3.2
kernel so for the time being, I've reverted to that kernel.
This is a production firewall system so I can't easily make it accessible for debugging, but I'll be happy to do test runs for any debugging kernels you may provide.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1251946/+subscriptions
References
