kernel-packages team mailing list archive
-
kernel-packages team
-
Mailing list archive
-
Message #06149
[Bug 1201869] Re: poor networking throughput across an OpenStack Neutron router on 3.5/3.8 kernels
Looks like with these patch we get the following crash:
PID: 6752 TASK: ffff8817d41e1700 CPU: 13 COMMAND: "ip"
#0 [ffff8817d2be33f0] machine_kexec at ffffffff8103bbda
#1 [ffff8817d2be3460] crash_kexec at ffffffff810bc028
#2 [ffff8817d2be3530] oops_end at ffffffff8169f098
#3 [ffff8817d2be3560] no_context at ffffffff8168436f
#4 [ffff8817d2be35b0] __bad_area_nosemaphore at ffffffff81684551
#5 [ffff8817d2be3610] bad_area at ffffffff816845ca
#6 [ffff8817d2be3640] do_page_fault at ffffffff816a1f54
#7 [ffff8817d2be3750] page_fault at ffffffff8169e4e5
[exception RIP: netif_carrier_off+9]
RIP: ffffffff8159f379 RSP: ffff8817d2be3808 RFLAGS: 00010292
RAX: 0000000000000001 RBX: ffff8817f3e36000 RCX: ffffffff81ca9948
RDX: 0000000000000000 RSI: 0000000000000282 RDI: 0000000000000000
RBP: ffff8817d2be3808 R8: ffff880c0fcee400 R9: ffff8817d2be38e8
R10: 0000000000001a60 R11: 0000000000000246 R12: ffff8817f3e36000
R13: ffff8817d2be3858 R14: ffff8817fd32be00 R15: 0000000000000000
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#8 [ffff8817d2be3810] veth_close at ffffffffa02221a2 [veth]
#9 [ffff8817d2be3830] __dev_close_many at ffffffff8157e3be
#10 [ffff8817d2be3850] dev_close_many at ffffffff8157e500
#11 [ffff8817d2be3890] rollback_registered_many at ffffffff8157e658
#12 [ffff8817d2be38c0] unregister_netdevice_many at ffffffff8157e7eb
#13 [ffff8817d2be38e0] rtnl_dellink at ffffffff8159148b
#14 [ffff8817d2be3a30] rtnetlink_rcv_msg at ffffffff81591be4
#15 [ffff8817d2be3ab0] netlink_rcv_skb at ffffffff815aab39
#16 [ffff8817d2be3ae0] rtnetlink_rcv at ffffffff8158fa35
#17 [ffff8817d2be3b00] netlink_unicast at ffffffff815aa4ad
#18 [ffff8817d2be3b50] netlink_sendmsg at ffffffff815aa810
#19 [ffff8817d2be3be0] sock_sendmsg at ffffffff81568e37
#20 [ffff8817d2be3d60] ___sys_sendmsg at ffffffff8156adcc
#21 [ffff8817d2be3f00] __sys_sendmsg at ffffffff8156c7b9
#22 [ffff8817d2be3f70] sys_sendmsg at ffffffff8156c819
#23 [ffff8817d2be3f80] system_call_fastpath at ffffffff816a65e9
RIP: 00007f7dd23fae30 RSP: 00007fff98b25f58 RFLAGS: 00010293
RAX: 000000000000002e RBX: ffffffff816a65e9 RCX: 0000000000000011
RDX: 0000000000000000 RSI: 00007fff98b21ef0 RDI: 0000000000000005
RBP: 0000000000000000 R8: 0000000000000000 R9: 0000000000000000
R10: 00007fff98b21c60 R11: 0000000000000246 R12: ffffffff8156c819
R13: ffff8817d2be3f78 R14: 0000000000000000 R15: 00007fff98b25fe8
ORIG_RAX: 000000000000002e CS: 0033 SS: 002b
A patch already exists that addresses this crash:
Commit 2efd32ee1b60b0b31404ca47c1ce70e5a5d24ebc
Author: Eric Dumazet <edumazet@xxxxxxxxxx>
Date: Thu Jan 10 08:32:45 2013 +0000
veth: fix a NULL deref in netif_carrier_off
In commit d0e2c55e7c94 (veth: avoid a NULL deref in veth_stats_one)
we now clear the peer pointers in veth_dellink()
veth_close() must therefore make sure the peer pointer is set.
** Tags removed: verification-needed-quantal verification-needed-raring
** Tags added: verification-failed-quantal verification-failed-raring
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1201869
Title:
poor networking throughput across an OpenStack Neutron router on
3.5/3.8 kernels
Status in “linux” package in Ubuntu:
Fix Released
Status in “linux” source package in Quantal:
Fix Committed
Status in “linux” source package in Raring:
Fix Committed
Bug description:
SRU Justification:
Impact:
Users of the 3.5/3.8 kernel will have poor network throughput when using OpenStack Neutron depending on their setup.
Fix:
These patches are all in Ubuntu-3.9.0-7.15 / v3.9-rc1:
2681128f0ced8aa4e66f221197e183cc16d244fe
8093315a91340bca52549044975d8c7f673b28a1
d0e2c55e7c940a3ee91e9e23a2683b593690f1e9
Testcase:
Setup OpenStack Neutron. Test throughput between internal and external nodes.
The following explains an example vlan+namespace configuration:
Internal Node: [10.x.x.2]->eth2.123->br123->tap123->qr-123[10.x.x.1] <--- netns: qrouter-123
netns: qrouter-123 ---> qg-234[10.x.y.1]->tap234->br234->eth2.234->External Node[10.x.y.2]
Where:
1) tap123+qr-123 and tap234+qg-234 are veth pairs
2) qr-123 and qg-234 reside inside the qrouter-123 namespace
--
OpenStack Neutron does IP forwarding through a network namespace. A
veth pair is used to connect into the namespace. The veth pair appears
to be the bottleneck, independent of network namespace. In newer
versions of Linux (Ubuntu-3.9.0-7.15 / v3.9-rc1 and greater)
throughput is much higher by almost 3 times. For example with some
testing throughput is 3.5 Gbps in pre 3.9-rc1 versions and 9.1 Gbps
with these patches applied.
This has been confirmed on kernels from 3.5.x-3.8.x. (Quantal and
Raring lts backports)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1201869/+subscriptions
References