← Back to team overview

kernel-packages team mailing list archive

[Bug 1396979] Re: APM Mustang board kernel crashes when running Ceph

 

[11072.912028] skbuff: skb_over_panic: text:ffffffbffc002578 len:2994 put:1514 head:ffffffc17a810000 data:ffffffc17a810064 tail:0xc16 end:0x680 dev:<NULL>
[11072.925544] Kernel panic - not syncing: BUG!
[11072.929792] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.16.0-24-generic #32-Ubuntu
[11072.937321] Call trace:
[11072.939759] [<ffffffc0000899f4>] dump_backtrace+0x0/0x174
[11072.945129] [<ffffffc000089b84>] show_stack+0x1c/0x28
[11072.950154] [<ffffffc0007316e8>] dump_stack+0x74/0x94
[11072.955178] [<ffffffc00072f05c>] panic+0xe8/0x228
[11072.959857] [<ffffffc000623e20>] skb_panic+0x94/0xa4
[11072.964794] [<ffffffc0006264c4>] skb_put+0xa8/0xac
[11072.969565] [<ffffffbffc002574>] xgene_enet_process_ring+0xdc/0x38c [xgene_enet]
[11072.976922] [<ffffffbffc00292c>] xgene_enet_napi+0x28/0x5c [xgene_enet]
[11072.983503] [<ffffffc000636e68>] net_rx_action+0x1a4/0x238
[11072.988959] [<ffffffc0000b2db8>] __do_softirq+0x124/0x284
[11072.994329] [<ffffffc0000b31a8>] irq_exit+0x94/0xc8
[11072.999181] [<ffffffc0000858b0>] handle_IRQ+0x54/0xd0
[11073.004204] [<ffffffc0000812f0>] gic_handle_irq+0x44/0x88
[11073.009574] Exception stack(0xffffffc3f01dfe00 to 0xffffffc3f01dff20)
[11073.015981] fe00: f01dc000 ffffffc3 00b7f308 ffffffc0 f01dff40 ffffffc3 00086290 ffffffc0
[11073.024116] fe20: 000f26f4 ffffffc0 00000000 00000000 00c26f80 ffffffc0 00ae5260 ffffffc0
[11073.032251] fe40: 00ae4000 ffffffc0 00106fc3 00000001 25c17d03 00000002 0a24a0c1 00000000
[11073.040385] fe60: 00000018 00000000 f01dfd60 ffffffc3 00000001 00000000 000a13fc 00000000
[11073.048520] fe80: 00000003 00000000 00000008 00000000 00000000 00000000 ab365590 0000007f
[11073.056654] fea0: 00205ddc ffffffc0 ab2df664 0000007f 5a8dc000 00000055 f01dc000 ffffffc3
[11073.064789] fec0: 00b7f308 ffffffc0 00b83fa0 ffffffc0 00b63000 ffffffc0 00000000 00000000
[11073.072923] fee0: 00000000 00000000 0007b000 00000040 00abe000 ffffffc0 007420d0 ffffffc0
[11073.081058] ff00: ffffffed 00000000 f01dff40 ffffffc3 0008628c ffffffc0 f01dff40 ffffffc3
[11073.089192] [<ffffffc000084da0>] el1_irq+0x60/0xc0
[11073.093959] [<ffffffc0000f26f0>] cpu_startup_entry+0x258/0x2f0
[11073.099761] [<ffffffc000090d64>] secondary_start_kernel+0x10c/0x11c
[11073.105997] CPU3: stopping
[11073.108693] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 3.16.0-24-generic #32-Ubuntu
[11073.116223] Call trace:
[11073.118659] [<ffffffc0000899f4>] dump_backtrace+0x0/0x174
[11073.124029] [<ffffffc000089b84>] show_stack+0x1c/0x28
[11073.129053] [<ffffffc0007316e8>] dump_stack+0x74/0x94
[11073.134077] [<ffffffc000091318>] handle_IPI+0x194/0x1ac
[11073.139274] [<ffffffc00008132c>] gic_handle_irq+0x80/0x88
[11073.144644] Exception stack(0xffffffc3f01efe00 to 0xffffffc3f01eff20)
[11073.151051] fe00: f01ec000 ffffffc3 00b7f308 ffffffc0 f01eff40 ffffffc3 00086290 ffffffc0
[11073.159186] fe20: 000f26f4 ffffffc0 00000000 00000000 00c26f80 ffffffc0 00ae5260 ffffffc0
[11073.167321] fe40: 00ae4000 ffffffc0 00106fd7 00000001 e81c7918 ffffffc3 0a24a0c1 00000000
[11073.175455] fe60: 00000018 00000000 f01efd60 ffffffc3 00000400 00000000 000003ff 00000000
[11073.183590] fe80: 00000001 00000000 00000000 ffffffff 00000000 00000000 4c7ab300 ffffffc0
[11073.191724] fea0: 00222a48 ffffffc0 ab2e1470 0000007f 0000000d 00000000 f01ec000 ffffffc3
[11073.199858] fec0: 00b7f308 ffffffc0 00b83fa0 ffffffc0 00b63000 ffffffc0 00000000 00000000
[11073.207993] fee0: 00000000 00000000 0007b000 00000040 00abe000 ffffffc0 007420d0 ffffffc0
[11073.216127] ff00: ffffffed 00000000 f01eff40 ffffffc3 0008628c ffffffc0 f01eff40 ffffffc3
[11073.224262] [<ffffffc000084da0>] el1_irq+0x60/0xc0
[11073.229028] [<ffffffc0000f26f0>] cpu_startup_entry+0x258/0x2f0
[11073.234829] [<ffffffc000090d64>] secondary_start_kernel+0x10c/0x11c
[11073.241063] CPU5: stopping
[11073.243759] CPU: 5 PID: 0 Comm: swapper/5 Not tainted 3.16.0-24-generic #32-Ubuntu
[11073.251289] Call trace:
[11073.253723] [<ffffffc0000899f4>] dump_backtrace+0x0/0x174
[11073.259094] [<ffffffc000089b84>] show_stack+0x1c/0x28
[11073.264118] [<ffffffc0007316e8>] dump_stack+0x74/0x94
[11073.269142] [<ffffffc000091318>] handle_IPI+0x194/0x1ac
[11073.274338] [<ffffffc00008132c>] gic_handle_irq+0x80/0x88
[11073.279707] Exception stack(0xffffffc3f01f7e00 to 0xffffffc3f01f7f20)
[11073.286114] 7e00: f01f4000 ffffffc3 00b7f308 ffffffc0 f01f7f40 ffffffc3 00086290 ffffffc0
[11073.294249] 7e20: 000f26f4 ffffffc0 00000000 00000000 00c26f80 ffffffc0 00ae5260 ffffffc0
[11073.302384] 7e40: 00ae4000 ffffffc0 00106fd7 00000001 fff972b0 ffffffc3 0a24a0c1 00000000
[11073.310519] 7e60: 00000018 00000000 f01f7d60 ffffffc3 00000400 00000000 000003ff 00000000
[11073.318653] 7e80: 00000000 00010000 00000000 00000000 00000000 00000000 00000000 00000000
[11073.326788] 7ea0: 00000000 00000000 07ffffff 00000000 00000000 00000000 f01f4000 ffffffc3
[11073.334922] 7ec0: 00b7f308 ffffffc0 00b83fa0 ffffffc0 00b63000 ffffffc0 00000000 00000000
[11073.343057] 7ee0: 00000000 00000000 0007b000 00000040 00abe000 ffffffc0 007420d0 ffffffc0
[11073.351191] 7f00: ffffffed 00000000 f01f7f40 ffffffc3 0008628c ffffffc0 f01f7f40 ffffffc3
[11073.359326] [<ffffffc000084da0>] el1_irq+0x60/0xc0
[11073.364092] [<ffffffc0000f26f0>] cpu_startup_entry+0x258/0x2f0
[11073.369894] [<ffffffc000090d64>] secondary_start_kernel+0x10c/0x11c
[11073.376127] CPU4: stopping
[11073.378821] CPU: 4 PID: 0 Comm: swapper/4 Not tainted 3.16.0-24-generic #32-Ubuntu
[11073.386350] Call trace:
[11073.388785] [<ffffffc0000899f4>] dump_backtrace+0x0/0x174
[11073.394156] [<ffffffc000089b84>] show_stack+0x1c/0x28
[11073.399179] [<ffffffc0007316e8>] dump_stack+0x74/0x94
[11073.404203] [<ffffffc000091318>] handle_IPI+0x194/0x1ac
[11073.409399] [<ffffffc00008132c>] gic_handle_irq+0x80/0x88
[11073.414768] Exception stack(0xffffffc3f01f3e00 to 0xffffffc3f01f3f20)
[11073.421175] 3e00: f01f0000 ffffffc3 00b7f308 ffffffc0 f01f3f40 ffffffc3 00086290 ffffffc0
[11073.429310] 3e20: 000f26f4 ffffffc0 00000000 00000000 00c26f80 ffffffc0 00ae5260 ffffffc0
[11073.437445] 3e40: 00ae4000 ffffffc0 00106fd2 00000001 25c17d03 00000002 0a24a0c1 00000000
[11073.445580] 3e60: 00000018 00000000 f01f3d60 ffffffc3 00000400 00000000 000003ff 00000000
[11073.453715] 3e80: 00000005 00000000 000000b9 00000000 00000000 00000000 ab365590 0000007f
[11073.461849] 3ea0: 0011de94 ffffffc0 ab6ee2dc 0000007f 5a8dc000 00000055 f01f0000 ffffffc3
[11073.469984] 3ec0: 00b7f308 ffffffc0 00b83fa0 ffffffc0 00b63000 ffffffc0 00000000 00000000
[11073.478118] 3ee0: 00000000 00000000 0007b000 00000040 00abe000 ffffffc0 007420d0 ffffffc0
[11073.486253] 3f00: ffffffed 00000000 f01f3f40 ffffffc3 0008628c ffffffc0 f01f3f40 ffffffc3
[11073.494387] [<ffffffc000084da0>] el1_irq+0x60/0xc0
[11073.499152] [<ffffffc0000f26f0>] cpu_startup_entry+0x258/0x2f0
[11073.504953] [<ffffffc000090d64>] secondary_start_kernel+0x10c/0x11c
[11073.511187] CPU2: stopping
[11073.513880] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 3.16.0-24-generic #32-Ubuntu
[11073.521410] Call trace:
[11073.523843] [<ffffffc0000899f4>] dump_backtrace+0x0/0x174
[11073.529214] [<ffffffc000089b84>] show_stack+0x1c/0x28
[11073.534237] [<ffffffc0007316e8>] dump_stack+0x74/0x94
[11073.539261] [<ffffffc000091318>] handle_IPI+0x194/0x1ac
[11073.544457] [<ffffffc00008132c>] gic_handle_irq+0x80/0x88
[11073.549826] Exception stack(0xffffffc3f01ebe00 to 0xffffffc3f01ebf20)
[11073.556233] be00: f01e8000 ffffffc3 00b7f308 ffffffc0 f01ebf40 ffffffc3 00086290 ffffffc0
[11073.564368] be20: 000f26f4 ffffffc0 00000000 00000000 00c26f80 ffffffc0 00ae5260 ffffffc0
[11073.572503] be40: 00ae4000 ffffffc0 00106fd5 00000001 25c17d03 00000002 0a24a0c1 00000000
[11073.580637] be60: 00000018 00000000 f01ebd60 ffffffc3 00000400 00000000 00000400 00000000
[11073.588772] be80: 6b736964 6972772b 65206574 20293931 63203476 65727275 ab365590 0000007f
[11073.596906] bea0: 0011de94 ffffffc0 ab6ee2dc 0000007f 0000000d 00000000 f01e8000 ffffffc3
[11073.605041] bec0: 00b7f308 ffffffc0 00b83fa0 ffffffc0 00b63000 ffffffc0 00000000 00000000
[11073.613175] bee0: 00000000 00000000 0007b000 00000040 00abe000 ffffffc0 007420d0 ffffffc0
[11073.621309] bf00: ffffffed 00000000 f01ebf40 ffffffc3 0008628c ffffffc0 f01ebf40 ffffffc3
[11073.629444] [<ffffffc000084da0>] el1_irq+0x60/0xc0
[11073.634209] [<ffffffc0000f26f0>] cpu_startup_entry+0x258/0x2f0
[11073.640011] [<ffffffc000090d64>] secondary_start_kernel+0x10c/0x11c
[11073.646245] CPU7: stopping
[11073.648941] CPU: 7 PID: 0 Comm: swapper/7 Not tainted 3.16.0-24-generic #32-Ubuntu
[11073.656470] Call trace:
[11073.658904] [<ffffffc0000899f4>] dump_backtrace+0x0/0x174
[11073.664275] [<ffffffc000089b84>] show_stack+0x1c/0x28
[11073.669299] [<ffffffc0007316e8>] dump_stack+0x74/0x94
[11073.674323] [<ffffffc000091318>] handle_IPI+0x194/0x1ac
[11073.679519] [<ffffffc00008132c>] gic_handle_irq+0x80/0x88
[11073.684888] Exception stack(0xffffffc3f01ffe00 to 0xffffffc3f01fff20)
[11073.691295] fe00: f01fc000 ffffffc3 00b7f308 ffffffc0 f01fff40 ffffffc3 00086290 ffffffc0
[11073.699430] fe20: 000f26f4 ffffffc0 00000000 00000000 00c26f80 ffffffc0 00ae5260 ffffffc0
[11073.707565] fe40: 00ae4000 ffffffc0 00106fd2 00000001 25c17d03 00000002 0a24a0c1 00000000
[11073.715700] fe60: 00000018 00000000 f01ffd60 ffffffc3 00000400 00000000 000003ff 00000000
[11073.723835] fe80: 00000010 00000000 00000040 00000000 39a67338 00000000 ab365590 0000007f
[11073.731969] fea0: 0011de94 ffffffc0 ab6ee2dc 0000007f 00006b00 00000000 f01fc000 ffffffc3
[11073.740104] fec0: 00b7f308 ffffffc0 00b83fa0 ffffffc0 00b63000 ffffffc0 00000000 00000000
[11073.748238] fee0: 00000000 00000000 0007b000 00000040 00abe000 ffffffc0 007420d0 ffffffc0
[11073.756373] ff00: ffffffed 00000000 f01fff40 ffffffc3 0008628c ffffffc0 f01fff40 ffffffc3
[11073.764507] [<ffffffc000084da0>] el1_irq+0x60/0xc0
[11073.769272] [<ffffffc0000f26f0>] cpu_startup_entry+0x258/0x2f0
[11073.775074] [<ffffffc000090d64>] secondary_start_kernel+0x10c/0x11c
[11073.781307] CPU6: stopping
[11073.784000] CPU: 6 PID: 0 Comm: swapper/6 Not tainted 3.16.0-24-generic #32-Ubuntu
[11073.791530] Call trace:
[11073.793963] [<ffffffc0000899f4>] dump_backtrace+0x0/0x174
[11073.799334] [<ffffffc000089b84>] show_stack+0x1c/0x28
[11073.804358] [<ffffffc0007316e8>] dump_stack+0x74/0x94
[11073.809382] [<ffffffc000091318>] handle_IPI+0x194/0x1ac
[11073.814577] [<ffffffc00008132c>] gic_handle_irq+0x80/0x88
[11073.819947] Exception stack(0xffffffc3f01fbe00 to 0xffffffc3f01fbf20)
[11073.826353] be00: f01f8000 ffffffc3 00b7f308 ffffffc0 f01fbf40 ffffffc3 00086290 ffffffc0
[11073.834489] be20: 000f26f4 ffffffc0 00000000 00000000 00c26f80 ffffffc0 00ae5260 ffffffc0
[11073.842624] be40: 00ae4000 ffffffc0 00106fd2 00000001 25c17d03 00000002 0a24a0c1 00000000
[11073.850758] be60: 00000018 00000000 f01fbd60 ffffffc3 00000400 00000000 000003ff 00000000
[11073.858893] be80: 00000018 00000000 e8000000 00000003 00000001 00000000 8da0a916 003be674
[11073.867028] bea0: 0011de94 ffffffc0 ab6ee650 0000007f e31d3e50 0000007f f01f8000 ffffffc3
[11073.875162] bec0: 00b7f308 ffffffc0 00b83fa0 ffffffc0 00b63000 ffffffc0 00000000 00000000
[11073.883297] bee0: 00000000 00000000 0007b000 00000040 00abe000 ffffffc0 007420d0 ffffffc0
[11073.891431] bf00: ffffffed 00000000 f01fbf40 ffffffc3 0008628c ffffffc0 f01fbf40 ffffffc3
[11073.899565] [<ffffffc000084da0>] el1_irq+0x60/0xc0
[11073.904330] [<ffffffc0000f26f0>] cpu_startup_entry+0x258/0x2f0
[11073.910131] [<ffffffc000090d64>] secondary_start_kernel+0x10c/0x11c
[11073.916365] CPU0: stopping
[11073.919058] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.16.0-24-generic #32-Ubuntu
[11073.926588] Call trace:
[11073.929021] [<ffffffc0000899f4>] dump_backtrace+0x0/0x174
[11073.934392] [<ffffffc000089b84>] show_stack+0x1c/0x28
[11073.939415] [<ffffffc0007316e8>] dump_stack+0x74/0x94
[11073.944439] [<ffffffc000091318>] handle_IPI+0x194/0x1ac
[11073.949635] [<ffffffc00008132c>] gic_handle_irq+0x80/0x88
[11073.955004] Exception stack(0xffffffc000ac3dc0 to 0xffffffc000ac3ee0)
[11073.961411] 3dc0: 00ac0000 ffffffc0 00b7f308 ffffffc0 00ac3f00 ffffffc0 00086290 ffffffc0
[11073.969547] 3de0: 000f26f4 ffffffc0 00000000 00000000 00c26f80 ffffffc0 00ae5260 ffffffc0
[11073.977681] 3e00: 00ae4000 ffffffc0 00106fd6 00000001 25c17d03 00000002 0a24a0c1 00000000
[11073.985816] 3e20: 00000018 00000000 00ac3d20 ffffffc0 00000400 00000000 000003ff 00000000
[11073.993951] 3e40: 00000001 00000000 00000000 ffffffff 00000000 00000000 526c4700 ffffffc2
[11074.002085] 3e60: 00222a48 ffffffc0 ab2e1470 0000007f 0000000d 00000000 00ac0000 ffffffc0
[11074.010220] 3e80: 00b7f308 ffffffc0 00b83fa0 ffffffc0 00b63000 ffffffc0 00000000 00000000
[11074.018355] 3ea0: 00000000 00000000 0007b000 00000040 00abe000 ffffffc0 007420d0 ffffffc0
[11074.026489] 3ec0: ffffffed 00000000 00ac3f00 ffffffc0 0008628c ffffffc0 00ac3f00 ffffffc0
[11074.034624] [<ffffffc000084da0>] el1_irq+0x60/0xc0
[11074.039390] [<ffffffc0000f26f0>] cpu_startup_entry+0x258/0x2f0
[11074.045191] [<ffffffc00072cdfc>] rest_init+0x74/0x80
[11074.050130] [<ffffffc000a35950>] start_kernel+0x384/0x39c
[11074.055505] ---[ end Kernel panic - not syncing: BUG

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1396979

Title:
  APM Mustang board kernel crashes when running Ceph

Status in “linux” package in Ubuntu:
  New

Bug description:
  Hi,
  We've been running Ceph on 4 APM Mustang boards within Linaro, running stock Ubuntu Utopic (problem occurs with Trusty too).

  Essentially when I place the network under heavy load (by writing data
  to a Ceph filesystem backed by 3xOSDs), I experience kernel crashes.
  I've harvested a few backtraces from the serial port and will paste
  them below as separate comments.

  I think the problem lies in the xgene driver, but I don't know enough
  to resolve where this problem is happening.

  Cheers,
  --
  Steve

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1396979/+subscriptions


References