yahoo-eng-team team mailing list archive

[OpenStack Infra <1777598@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/576418
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5ddb2a4762eac52290cd0491881ec6ca15bb9217
Submitter: Zuul
Branch:    master

commit 5ddb2a4762eac52290cd0491881ec6ca15bb9217
Author: LIU Yulong <i@xxxxxxxxxxxx>
Date:   Fri Jun 15 16:02:36 2018 -0400

    Make L3 IP tc filter rate limit more accurate
    
    Currently the L3 agent qos extension does not set the mtu for
    tc filter rules, the default value is 2kb. Tc filter rules use
    such mtu will result in an inaccurate bandwidth.
    
    So in order to improve the precision, we set the mtu to 64kb.
    For the test results, you can read the bug description.
    
    For more information you can read the linux tc man page:
    [1] https://linux.die.net/man/8/tc-tbf
    [2] http://man7.org/linux/man-pages/man8/tc-police.8.html
    
    Closes-Bug: #1777598
    Change-Id: I7a167ec6139ccc55988b34f82080a182116d02a3


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1777598

Title:
  Inaccurate L3 QoS bandwidth

Status in neutron:
  Fix Released

Bug description:
  ENV:
  stable/queens (12.0.1)
  centos 7

  Steps:
  1. create a vm with floating IP 172.18.213.143
  2. create qos policy with ingress and egress bandwidth limit rule:
      burst 10240 rate 10240
      aka, a 10mbps rule
  3. run iperf3 in vm
  4. iperf3 connect the vm via floating IP 172.18.213.143

  iperf3 Testing:
  the iperf output:
  # iperf3 -c 172.18.213.143 -i 1
  Connecting to host 172.18.213.143, port 5201
  [  4] local 172.18.211.30 port 36980 connected to 172.18.213.143 port 5201
  [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
  [  4]   0.00-1.00   sec   611 KBytes  5.00 Mbits/sec   54   2.83 KBytes
  [  4]   1.00-2.00   sec   752 KBytes  6.16 Mbits/sec   53   4.24 KBytes
  [  4]   2.00-3.00   sec   898 KBytes  7.35 Mbits/sec   57   4.24 KBytes
  [  4]   3.00-4.00   sec   604 KBytes  4.95 Mbits/sec   49   5.66 KBytes
  [  4]   4.00-5.00   sec   236 KBytes  1.93 Mbits/sec   23   9.90 KBytes
  [  4]   5.00-6.00   sec   810 KBytes  6.64 Mbits/sec  154   4.24 KBytes
  [  4]   6.00-7.00   sec   127 KBytes  1.04 Mbits/sec   27   5.66 KBytes
  [  4]   7.00-8.00   sec   509 KBytes  4.17 Mbits/sec   42   2.83 KBytes
  [  4]   8.00-9.00   sec   636 KBytes  5.21 Mbits/sec   48   4.24 KBytes
  [  4]   9.00-10.00  sec   127 KBytes  1.04 Mbits/sec   20   5.66 KBytes
  - - - - - - - - - - - - - - - - - - - - - - - - -
  [ ID] Interval           Transfer     Bandwidth       Retr
  [  4]   0.00-10.00  sec  5.19 MBytes  4.35 Mbits/sec  527             sender
  [  4]   0.00-10.00  sec  4.90 MBytes  4.11 Mbits/sec                  receiver

  iperf Done.

  The inaccurate bandwidth is because of the inappropriate tc filter mtu, default value is 2kb.
  here is the filter:
  =================================
  filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1  (rule hit 5675 success 5675)
    match IP src 172.18.213.143/32 (success 5675 )
   police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
  ref 1 bind 1

   Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0)
  =================================
  filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1  (rule hit 33937 success 33937)
    match IP dst 172.18.213.143/32 (success 33937 )
   police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b
  ref 1 bind 1

   Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268)
  =================================

  Solution:
  set the mtu to 64kb
  =================================
  filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1  (rule hit 0 success 0)
    match IP dst 172.18.213.143/32 (success 0 )
   police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
  ref 1 bind 1

   Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  =================================
  filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1  (rule hit 0 success 0)
    match IP src 172.18.213.143/32 (success 0 )
   police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b
  ref 1 bind 1

   Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  =================================

  
  iperf testing:

  Connecting to host 172.18.213.143, port 5201
  [  4] local 172.18.211.30 port 37886 connected to 172.18.213.143 port 5201
  [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
  [  4]   0.00-1.00   sec  2.49 MBytes  20.8 Mbits/sec  146   28.3 KBytes
  [  4]   1.00-2.00   sec  1.30 MBytes  10.9 Mbits/sec  119   7.07 KBytes
  [  4]   2.00-3.00   sec  1.24 MBytes  10.4 Mbits/sec   77   18.4 KBytes
  [  4]   3.00-4.00   sec  1.12 MBytes  9.38 Mbits/sec   72   19.8 KBytes
  [  4]   4.00-5.00   sec  1.12 MBytes  9.38 Mbits/sec   77   33.9 KBytes
  [  4]   5.00-6.00   sec  1.12 MBytes  9.38 Mbits/sec  100   39.6 KBytes
  [  4]   6.00-7.00   sec  1.18 MBytes  9.90 Mbits/sec  145   42.4 KBytes
  [  4]   7.00-8.00   sec  1.24 MBytes  10.4 Mbits/sec  141   32.5 KBytes
  [  4]   8.00-9.00   sec  1.43 MBytes  12.0 Mbits/sec  126   42.4 KBytes
  [  4]   9.00-10.00  sec  1.12 MBytes  9.38 Mbits/sec   80   24.0 KBytes
  - - - - - - - - - - - - - - - - - - - - - - - - -
  [ ID] Interval           Transfer     Bandwidth       Retr
  [  4]   0.00-10.00  sec  13.4 MBytes  11.2 Mbits/sec  1083             sender
  [  4]   0.00-10.00  sec  13.0 MBytes  10.9 Mbits/sec                  receiver

  iperf Done.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1777598/+subscriptions