yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #73374
[Bug 1777598] [NEW] Inaccurate L3 QoS bandwidth
Public bug reported:
ENV:
stable/queens (12.0.1)
centos 7
Steps:
1. create a vm with floating IP 172.18.213.143
2. create qos policy with ingress and egress bandwidth limit rule:
burst 10240 rate 10240
aka, a 10mbps rule
3. run iperf3 in vm
4. iperf3 connect the vm via floating IP 172.18.213.143
iperf3 Testing:
the iperf output:
# iperf3 -c 172.18.213.143 -i 1
Connecting to host 172.18.213.143, port 5201
[ 4] local 172.18.211.30 port 36980 connected to 172.18.213.143 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 611 KBytes 5.00 Mbits/sec 54 2.83 KBytes
[ 4] 1.00-2.00 sec 752 KBytes 6.16 Mbits/sec 53 4.24 KBytes
[ 4] 2.00-3.00 sec 898 KBytes 7.35 Mbits/sec 57 4.24 KBytes
[ 4] 3.00-4.00 sec 604 KBytes 4.95 Mbits/sec 49 5.66 KBytes
[ 4] 4.00-5.00 sec 236 KBytes 1.93 Mbits/sec 23 9.90 KBytes
[ 4] 5.00-6.00 sec 810 KBytes 6.64 Mbits/sec 154 4.24 KBytes
[ 4] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 27 5.66 KBytes
[ 4] 7.00-8.00 sec 509 KBytes 4.17 Mbits/sec 42 2.83 KBytes
[ 4] 8.00-9.00 sec 636 KBytes 5.21 Mbits/sec 48 4.24 KBytes
[ 4] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 20 5.66 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 5.19 MBytes 4.35 Mbits/sec 527 sender
[ 4] 0.00-10.00 sec 4.90 MBytes 4.11 Mbits/sec receiver
iperf Done.
The inaccurate bandwidth is because of the inappropriate tc filter mtu, default value is 2kb.
here is the filter:
=================================
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 5675 success 5675)
match IP src 172.18.213.143/32 (success 5675 )
police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
ref 1 bind 1
Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0)
=================================
filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 33937 success 33937)
match IP dst 172.18.213.143/32 (success 33937 )
police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
ref 1 bind 1
Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268)
=================================
Solution:
set the mtu to 64kb
=================================
filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0)
match IP dst 172.18.213.143/32 (success 0 )
police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
ref 1 bind 1
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
=================================
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0)
match IP src 172.18.213.143/32 (success 0 )
police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
ref 1 bind 1
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
=================================
iperf testing:
Connecting to host 172.18.213.143, port 5201
[ 4] local 172.18.211.30 port 37886 connected to 172.18.213.143 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 2.49 MBytes 20.8 Mbits/sec 146 28.3 KBytes
[ 4] 1.00-2.00 sec 1.30 MBytes 10.9 Mbits/sec 119 7.07 KBytes
[ 4] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 77 18.4 KBytes
[ 4] 3.00-4.00 sec 1.12 MBytes 9.38 Mbits/sec 72 19.8 KBytes
[ 4] 4.00-5.00 sec 1.12 MBytes 9.38 Mbits/sec 77 33.9 KBytes
[ 4] 5.00-6.00 sec 1.12 MBytes 9.38 Mbits/sec 100 39.6 KBytes
[ 4] 6.00-7.00 sec 1.18 MBytes 9.90 Mbits/sec 145 42.4 KBytes
[ 4] 7.00-8.00 sec 1.24 MBytes 10.4 Mbits/sec 141 32.5 KBytes
[ 4] 8.00-9.00 sec 1.43 MBytes 12.0 Mbits/sec 126 42.4 KBytes
[ 4] 9.00-10.00 sec 1.12 MBytes 9.38 Mbits/sec 80 24.0 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 13.4 MBytes 11.2 Mbits/sec 1083 sender
[ 4] 0.00-10.00 sec 13.0 MBytes 10.9 Mbits/sec receiver
iperf Done.
** Affects: neutron
Importance: Undecided
Assignee: LIU Yulong (dragon889)
Status: New
** Description changed:
ENV:
stable/queens (12.0.1)
centos 7
Steps:
- 1. create a vm with floating IP 1.1.1.1
+ 1. create a vm with floating IP 172.18.213.143
2. create qos policy with ingress and egress bandwidth limit rule:
- burst 10240 rate 10240
- aka, a 10mbps rule
+ burst 10240 rate 10240
+ aka, a 10mbps rule
3. run iperf3 in vm
-
+ 4. iperf3 connect the vm via floating IP 172.18.213.143
iperf3 Testing:
the iperf output:
# iperf3 -c 172.18.213.143 -i 1
Connecting to host 172.18.213.143, port 5201
[ 4] local 172.18.211.30 port 36980 connected to 172.18.213.143 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
- [ 4] 0.00-1.00 sec 611 KBytes 5.00 Mbits/sec 54 2.83 KBytes
- [ 4] 1.00-2.00 sec 752 KBytes 6.16 Mbits/sec 53 4.24 KBytes
- [ 4] 2.00-3.00 sec 898 KBytes 7.35 Mbits/sec 57 4.24 KBytes
- [ 4] 3.00-4.00 sec 604 KBytes 4.95 Mbits/sec 49 5.66 KBytes
- [ 4] 4.00-5.00 sec 236 KBytes 1.93 Mbits/sec 23 9.90 KBytes
- [ 4] 5.00-6.00 sec 810 KBytes 6.64 Mbits/sec 154 4.24 KBytes
- [ 4] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 27 5.66 KBytes
- [ 4] 7.00-8.00 sec 509 KBytes 4.17 Mbits/sec 42 2.83 KBytes
- [ 4] 8.00-9.00 sec 636 KBytes 5.21 Mbits/sec 48 4.24 KBytes
- [ 4] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 20 5.66 KBytes
+ [ 4] 0.00-1.00 sec 611 KBytes 5.00 Mbits/sec 54 2.83 KBytes
+ [ 4] 1.00-2.00 sec 752 KBytes 6.16 Mbits/sec 53 4.24 KBytes
+ [ 4] 2.00-3.00 sec 898 KBytes 7.35 Mbits/sec 57 4.24 KBytes
+ [ 4] 3.00-4.00 sec 604 KBytes 4.95 Mbits/sec 49 5.66 KBytes
+ [ 4] 4.00-5.00 sec 236 KBytes 1.93 Mbits/sec 23 9.90 KBytes
+ [ 4] 5.00-6.00 sec 810 KBytes 6.64 Mbits/sec 154 4.24 KBytes
+ [ 4] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 27 5.66 KBytes
+ [ 4] 7.00-8.00 sec 509 KBytes 4.17 Mbits/sec 42 2.83 KBytes
+ [ 4] 8.00-9.00 sec 636 KBytes 5.21 Mbits/sec 48 4.24 KBytes
+ [ 4] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 20 5.66 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 5.19 MBytes 4.35 Mbits/sec 527 sender
[ 4] 0.00-10.00 sec 4.90 MBytes 4.11 Mbits/sec receiver
iperf Done.
-
-
- The inaccurate bandwidth is because of the inappropriate tc filter policy,
- default value 2kb.
+ The inaccurate bandwidth is because of the inappropriate tc filter mtu, default value is 2kb.
here is the filter:
+ =================================
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 5675 success 5675)
- match IP src 172.18.213.143/32 (success 5675 )
- police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
+ match IP src 172.18.213.143/32 (success 5675 )
+ police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
ref 1 bind 1
- Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0)
-
-
+ Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0)
+ =================================
filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 33937 success 33937)
- match IP dst 172.18.213.143/32 (success 33937 )
- police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
+ match IP dst 172.18.213.143/32 (success 33937 )
+ police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
ref 1 bind 1
- Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268)
-
+ Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268)
+ =================================
Solution:
set the mtu to 64kb
-
+ =================================
filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0)
- match IP dst 172.18.213.143/32 (success 0 )
- police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
+ match IP dst 172.18.213.143/32 (success 0 )
+ police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
ref 1 bind 1
- Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
-
+ Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
+ =================================
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0)
- match IP src 172.18.213.143/32 (success 0 )
- police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
+ match IP src 172.18.213.143/32 (success 0 )
+ police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
ref 1 bind 1
- Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
+ Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
+ =================================
+
iperf testing:
Connecting to host 172.18.213.143, port 5201
[ 4] local 172.18.211.30 port 37886 connected to 172.18.213.143 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
- [ 4] 0.00-1.00 sec 2.49 MBytes 20.8 Mbits/sec 146 28.3 KBytes
- [ 4] 1.00-2.00 sec 1.30 MBytes 10.9 Mbits/sec 119 7.07 KBytes
- [ 4] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 77 18.4 KBytes
- [ 4] 3.00-4.00 sec 1.12 MBytes 9.38 Mbits/sec 72 19.8 KBytes
- [ 4] 4.00-5.00 sec 1.12 MBytes 9.38 Mbits/sec 77 33.9 KBytes
- [ 4] 5.00-6.00 sec 1.12 MBytes 9.38 Mbits/sec 100 39.6 KBytes
- [ 4] 6.00-7.00 sec 1.18 MBytes 9.90 Mbits/sec 145 42.4 KBytes
- [ 4] 7.00-8.00 sec 1.24 MBytes 10.4 Mbits/sec 141 32.5 KBytes
- [ 4] 8.00-9.00 sec 1.43 MBytes 12.0 Mbits/sec 126 42.4 KBytes
- [ 4] 9.00-10.00 sec 1.12 MBytes 9.38 Mbits/sec 80 24.0 KBytes
+ [ 4] 0.00-1.00 sec 2.49 MBytes 20.8 Mbits/sec 146 28.3 KBytes
+ [ 4] 1.00-2.00 sec 1.30 MBytes 10.9 Mbits/sec 119 7.07 KBytes
+ [ 4] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 77 18.4 KBytes
+ [ 4] 3.00-4.00 sec 1.12 MBytes 9.38 Mbits/sec 72 19.8 KBytes
+ [ 4] 4.00-5.00 sec 1.12 MBytes 9.38 Mbits/sec 77 33.9 KBytes
+ [ 4] 5.00-6.00 sec 1.12 MBytes 9.38 Mbits/sec 100 39.6 KBytes
+ [ 4] 6.00-7.00 sec 1.18 MBytes 9.90 Mbits/sec 145 42.4 KBytes
+ [ 4] 7.00-8.00 sec 1.24 MBytes 10.4 Mbits/sec 141 32.5 KBytes
+ [ 4] 8.00-9.00 sec 1.43 MBytes 12.0 Mbits/sec 126 42.4 KBytes
+ [ 4] 9.00-10.00 sec 1.12 MBytes 9.38 Mbits/sec 80 24.0 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 13.4 MBytes 11.2 Mbits/sec 1083 sender
[ 4] 0.00-10.00 sec 13.0 MBytes 10.9 Mbits/sec receiver
iperf Done.
** Changed in: neutron
Assignee: (unassigned) => LIU Yulong (dragon889)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1777598
Title:
Inaccurate L3 QoS bandwidth
Status in neutron:
New
Bug description:
ENV:
stable/queens (12.0.1)
centos 7
Steps:
1. create a vm with floating IP 172.18.213.143
2. create qos policy with ingress and egress bandwidth limit rule:
burst 10240 rate 10240
aka, a 10mbps rule
3. run iperf3 in vm
4. iperf3 connect the vm via floating IP 172.18.213.143
iperf3 Testing:
the iperf output:
# iperf3 -c 172.18.213.143 -i 1
Connecting to host 172.18.213.143, port 5201
[ 4] local 172.18.211.30 port 36980 connected to 172.18.213.143 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 611 KBytes 5.00 Mbits/sec 54 2.83 KBytes
[ 4] 1.00-2.00 sec 752 KBytes 6.16 Mbits/sec 53 4.24 KBytes
[ 4] 2.00-3.00 sec 898 KBytes 7.35 Mbits/sec 57 4.24 KBytes
[ 4] 3.00-4.00 sec 604 KBytes 4.95 Mbits/sec 49 5.66 KBytes
[ 4] 4.00-5.00 sec 236 KBytes 1.93 Mbits/sec 23 9.90 KBytes
[ 4] 5.00-6.00 sec 810 KBytes 6.64 Mbits/sec 154 4.24 KBytes
[ 4] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 27 5.66 KBytes
[ 4] 7.00-8.00 sec 509 KBytes 4.17 Mbits/sec 42 2.83 KBytes
[ 4] 8.00-9.00 sec 636 KBytes 5.21 Mbits/sec 48 4.24 KBytes
[ 4] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 20 5.66 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 5.19 MBytes 4.35 Mbits/sec 527 sender
[ 4] 0.00-10.00 sec 4.90 MBytes 4.11 Mbits/sec receiver
iperf Done.
The inaccurate bandwidth is because of the inappropriate tc filter mtu, default value is 2kb.
here is the filter:
=================================
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 5675 success 5675)
match IP src 172.18.213.143/32 (success 5675 )
police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
ref 1 bind 1
Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0)
=================================
filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 33937 success 33937)
match IP dst 172.18.213.143/32 (success 33937 )
police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
ref 1 bind 1
Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268)
=================================
Solution:
set the mtu to 64kb
=================================
filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0)
match IP dst 172.18.213.143/32 (success 0 )
police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
ref 1 bind 1
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
=================================
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0)
match IP src 172.18.213.143/32 (success 0 )
police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
ref 1 bind 1
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
=================================
iperf testing:
Connecting to host 172.18.213.143, port 5201
[ 4] local 172.18.211.30 port 37886 connected to 172.18.213.143 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 2.49 MBytes 20.8 Mbits/sec 146 28.3 KBytes
[ 4] 1.00-2.00 sec 1.30 MBytes 10.9 Mbits/sec 119 7.07 KBytes
[ 4] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 77 18.4 KBytes
[ 4] 3.00-4.00 sec 1.12 MBytes 9.38 Mbits/sec 72 19.8 KBytes
[ 4] 4.00-5.00 sec 1.12 MBytes 9.38 Mbits/sec 77 33.9 KBytes
[ 4] 5.00-6.00 sec 1.12 MBytes 9.38 Mbits/sec 100 39.6 KBytes
[ 4] 6.00-7.00 sec 1.18 MBytes 9.90 Mbits/sec 145 42.4 KBytes
[ 4] 7.00-8.00 sec 1.24 MBytes 10.4 Mbits/sec 141 32.5 KBytes
[ 4] 8.00-9.00 sec 1.43 MBytes 12.0 Mbits/sec 126 42.4 KBytes
[ 4] 9.00-10.00 sec 1.12 MBytes 9.38 Mbits/sec 80 24.0 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 13.4 MBytes 11.2 Mbits/sec 1083 sender
[ 4] 0.00-10.00 sec 13.0 MBytes 10.9 Mbits/sec receiver
iperf Done.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1777598/+subscriptions
Follow ups
