kernel-packages team mailing list archive

Thread
Date
[Bug 1514363] [NEW] kernel crash in ipv4_dst_destroy

To: kernel-packages@xxxxxxxxxxxxxxxxxxx
From: Tom Eichhorn <tom@xxxxxxxxxxxx>
Date: Mon, 09 Nov 2015 09:16:30 -0000
Reply-to: Bug 1514363 <1514363@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Hi,

we have seen a GPF resulting in ipv4_dst_destroy:

messages-2015-11-08:Nov  8 20:23:05 bbk130129 kernel: [205505.064821] device tap406dbc6f-a8 left promiscuous mode
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205505.534610] general protection fault: 0000 [#1] SMP
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205505.578139] Modules linked in: kvm_intel kvm vport_vxlan vport_gre gre openvswitch dm_multipath scsi_dh nbd sch_fq_codel xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat bridge ebtable_filter ebtables ipmi_devintf btrfs xor raid6_pq ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs libcrc32c vhost_net vhost macvtap macvlan ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_limit xt_multiport xt_CT iptable_raw xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables mlx4_en(OE) mlx4_core(OE) mlx_compat(OE) isofs ipmi_ssif intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd sb_edac edac_core lpc_ich hpilo ioatdma ipmi_si 8250_fintek ipmi_msghandler wmi acpi_power_meter mac_hid shpchp bonding 8021q garp mrp stp llc lp parport nls_iso8859_1 ixgbe dca vxlan ip6_udp_tunnel tg3 udp_tunnel mpt3sas(OE) raid_class ptp configfs pps_core mdio scsi_transport_sas
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.275481] CPU: 42 PID: 258 Comm: rcuos/35 Tainted: G           OE  3.19.0-32-generic #37~14.04.1-Ubuntu
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.349044] Hardware name: HP ProLiant DL380 Gen9, BIOS P89 07/20/2015
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.404252] task: ffff882fa5823110 ti: ffff882fa5848000 task.ti: ffff882fa5848000
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.463760] RIP: 0010:[<ffffffff816de782>]  [<ffffffff816de782>] ipv4_dst_destroy+0x42/0x80
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.527658] RSP: 0018:ffff882fa584bde8  EFLAGS: 00010246
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.578843] RAX: dead000000200200 RBX: ffff882f1e961200 RCX: 0000000000000060
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.636886] RDX: dead000000100100 RSI: 00000000fffffe01 RDI: ffffffff81fc5d84
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.696883] RBP: ffff882fa584bdf8 R08: ffff882f1e961200 R09: 00000001802a000b
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.756193] R10: ffffffff816b30f5 R11: ffffea00bc7a5800 R12: 0000000000000000
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.814938] R13: 0000000000000001 R14: ffff882f1e961080 R15: ffff882fa5823110
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.878741] FS:  0000000000000000(0000) GS:ffff885fbf440000(0000) knlGS:0000000000000000
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.943046] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.997859] CR2: 00007ffd99ac3640 CR3: 0000000002c16000 CR4: 00000000001427e0
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.057616] Stack:
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.093614]  0000000000000000 ffff882f1e961200 ffff882fa584be18 ffffffff816b30d2
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.155762]  ffff882f1e961200 ffff882fbfcf4bc0 ffff882fa584be28 ffffffff816b33ae
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.217537]  ffff882fa584beb8 ffffffff810d542f ffff882fa5823110 ffff882fa584be78
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.281530] Call Trace:
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.321761]  [<ffffffff816b30d2>] dst_destroy+0x32/0xe0
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.373239]  [<ffffffff816b33ae>] dst_destroy_rcu+0xe/0x20
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.424174]  [<ffffffff810d542f>] rcu_nocb_kthread+0x19f/0x590
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.474698]  [<ffffffff810b4e30>] ? prepare_to_wait_event+0x110/0x110
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.530284]  [<ffffffff810d5290>] ? rcu_spawn_all_nocb_kthreads.part.54+0x160/0x160
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.589559]  [<ffffffff81093822>] kthread+0xd2/0xf0
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.635397]  [<ffffffff81093750>] ? kthread_create_on_node+0x1c0/0x1c0
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.691201]  [<ffffffff817b6d18>] ret_from_fork+0x58/0x90
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.741015]  [<ffffffff81093750>] ? kthread_create_on_node+0x1c0/0x1c0
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.798002] Code: 08 48 39 87 b0 00 00 00 74 4f 48 c7 c7 84 5d fc 81 e8 a3 82 0d 00 48 8b 83 b8 00 00 00 48 8b 93 b0 00 00 00 48 c7 c7 84 5d fc 81 <48> 89 42 08 48 89 10 48 b8 00 01 10 00 00 00 ad de 48 89 83 b0
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205508.001730]  RSP <ffff882fa584bde8>
messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205508.087352] ---[ end trace 645de9b56a152109 ]---

What has happened is that openstack has destroyed a VM and a network
interface has been removed from the kernel:

auth-2015-11-08.log:Nov  8 20:23:01 bbk130129 sudo:     nova : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/nova-rootwrap /etc/nova/rootwrap.conf mm-ctl --unbind-port 406dbc6f-a8fb-42f1-8f8d-a668698c7676
auth-2015-11-08.log:Nov  8 20:23:01 bbk130129 sudo: pam_unix(sudo:session): session opened for user root by (uid=0)
auth-2015-11-08.log:Nov  8 20:23:05 bbk130129 sudo: pam_unix(sudo:session): session closed for user root
auth-2015-11-08.log:Nov  8 20:23:05 bbk130129 sudo:     nova : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/nova-rootwrap /etc/nova/rootwrap.conf ip link delete tap406dbc6f-a8

Some googling had brought up http://lists.openwall.net/netdev/2014/01/22/29 and http://lists.openwall.net/netdev/2014/01/18/5,
which might here also be the problem.

I have not yet been able to reproduce this bug, but will update this bug
if it happens again.

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: linux-image-3.19.0-32-generic 3.19.0-32.37~14.04.1
ProcVersionSignature: Ubuntu 3.19.0-32.37~14.04.1-generic 3.19.8-ckt7
Uname: Linux 3.19.0-32-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.18
Architecture: amd64
Date: Mon Nov  9 09:09:04 2015
SourcePackage: linux-lts-vivid
UpgradeStatus: No upgrade log present (probably fresh install)

** Affects: linux-lts-vivid (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 apport-bug trusty

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-lts-vivid in Ubuntu.
https://bugs.launchpad.net/bugs/1514363

Title:
  kernel crash in ipv4_dst_destroy

Status in linux-lts-vivid package in Ubuntu:
  New

Bug description:
  Hi,

  we have seen a GPF resulting in ipv4_dst_destroy:

  messages-2015-11-08:Nov  8 20:23:05 bbk130129 kernel: [205505.064821] device tap406dbc6f-a8 left promiscuous mode
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205505.534610] general protection fault: 0000 [#1] SMP
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205505.578139] Modules linked in: kvm_intel kvm vport_vxlan vport_gre gre openvswitch dm_multipath scsi_dh nbd sch_fq_codel xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat bridge ebtable_filter ebtables ipmi_devintf btrfs xor raid6_pq ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs libcrc32c vhost_net vhost macvtap macvlan ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_limit xt_multiport xt_CT iptable_raw xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables mlx4_en(OE) mlx4_core(OE) mlx_compat(OE) isofs ipmi_ssif intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd sb_edac edac_core lpc_ich hpilo ioatdma ipmi_si 8250_fintek ipmi_msghandler wmi acpi_power_meter mac_hid shpchp bonding 8021q garp mrp stp llc lp parport nls_iso8859_1 ixgbe dca vxlan ip6_udp_tunnel tg3 udp_tunnel mpt3sas(OE) raid_class ptp configfs pps_core mdio scsi_transport_sas
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.275481] CPU: 42 PID: 258 Comm: rcuos/35 Tainted: G           OE  3.19.0-32-generic #37~14.04.1-Ubuntu
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.349044] Hardware name: HP ProLiant DL380 Gen9, BIOS P89 07/20/2015
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.404252] task: ffff882fa5823110 ti: ffff882fa5848000 task.ti: ffff882fa5848000
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.463760] RIP: 0010:[<ffffffff816de782>]  [<ffffffff816de782>] ipv4_dst_destroy+0x42/0x80
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.527658] RSP: 0018:ffff882fa584bde8  EFLAGS: 00010246
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.578843] RAX: dead000000200200 RBX: ffff882f1e961200 RCX: 0000000000000060
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.636886] RDX: dead000000100100 RSI: 00000000fffffe01 RDI: ffffffff81fc5d84
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.696883] RBP: ffff882fa584bdf8 R08: ffff882f1e961200 R09: 00000001802a000b
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.756193] R10: ffffffff816b30f5 R11: ffffea00bc7a5800 R12: 0000000000000000
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.814938] R13: 0000000000000001 R14: ffff882f1e961080 R15: ffff882fa5823110
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.878741] FS:  0000000000000000(0000) GS:ffff885fbf440000(0000) knlGS:0000000000000000
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.943046] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205506.997859] CR2: 00007ffd99ac3640 CR3: 0000000002c16000 CR4: 00000000001427e0
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.057616] Stack:
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.093614]  0000000000000000 ffff882f1e961200 ffff882fa584be18 ffffffff816b30d2
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.155762]  ffff882f1e961200 ffff882fbfcf4bc0 ffff882fa584be28 ffffffff816b33ae
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.217537]  ffff882fa584beb8 ffffffff810d542f ffff882fa5823110 ffff882fa584be78
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.281530] Call Trace:
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.321761]  [<ffffffff816b30d2>] dst_destroy+0x32/0xe0
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.373239]  [<ffffffff816b33ae>] dst_destroy_rcu+0xe/0x20
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.424174]  [<ffffffff810d542f>] rcu_nocb_kthread+0x19f/0x590
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.474698]  [<ffffffff810b4e30>] ? prepare_to_wait_event+0x110/0x110
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.530284]  [<ffffffff810d5290>] ? rcu_spawn_all_nocb_kthreads.part.54+0x160/0x160
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.589559]  [<ffffffff81093822>] kthread+0xd2/0xf0
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.635397]  [<ffffffff81093750>] ? kthread_create_on_node+0x1c0/0x1c0
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.691201]  [<ffffffff817b6d18>] ret_from_fork+0x58/0x90
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.741015]  [<ffffffff81093750>] ? kthread_create_on_node+0x1c0/0x1c0
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205507.798002] Code: 08 48 39 87 b0 00 00 00 74 4f 48 c7 c7 84 5d fc 81 e8 a3 82 0d 00 48 8b 83 b8 00 00 00 48 8b 93 b0 00 00 00 48 c7 c7 84 5d fc 81 <48> 89 42 08 48 89 10 48 b8 00 01 10 00 00 00 ad de 48 89 83 b0
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205508.001730]  RSP <ffff882fa584bde8>
  messages-2015-11-08:Nov  8 20:23:08 bbk130129 kernel: [205508.087352] ---[ end trace 645de9b56a152109 ]---

  What has happened is that openstack has destroyed a VM and a network
  interface has been removed from the kernel:

  auth-2015-11-08.log:Nov  8 20:23:01 bbk130129 sudo:     nova : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/nova-rootwrap /etc/nova/rootwrap.conf mm-ctl --unbind-port 406dbc6f-a8fb-42f1-8f8d-a668698c7676
  auth-2015-11-08.log:Nov  8 20:23:01 bbk130129 sudo: pam_unix(sudo:session): session opened for user root by (uid=0)
  auth-2015-11-08.log:Nov  8 20:23:05 bbk130129 sudo: pam_unix(sudo:session): session closed for user root
  auth-2015-11-08.log:Nov  8 20:23:05 bbk130129 sudo:     nova : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/nova-rootwrap /etc/nova/rootwrap.conf ip link delete tap406dbc6f-a8

  Some googling had brought up http://lists.openwall.net/netdev/2014/01/22/29 and http://lists.openwall.net/netdev/2014/01/18/5,
  which might here also be the problem.

  I have not yet been able to reproduce this bug, but will update this
  bug if it happens again.

  ProblemType: Bug
  DistroRelease: Ubuntu 14.04
  Package: linux-image-3.19.0-32-generic 3.19.0-32.37~14.04.1
  ProcVersionSignature: Ubuntu 3.19.0-32.37~14.04.1-generic 3.19.8-ckt7
  Uname: Linux 3.19.0-32-generic x86_64
  ApportVersion: 2.14.1-0ubuntu3.18
  Architecture: amd64
  Date: Mon Nov  9 09:09:04 2015
  SourcePackage: linux-lts-vivid
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-lts-vivid/+bug/1514363/+subscriptions
Follow ups

[Bug 1514363] Re: kernel crash in ipv4_dst_destroy
From: Launchpad Bug Tracker, 2015-11-09