yahoo-eng-team team mailing list archive

[Slawek Kaplonski <1948642@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When neutron-ovs-agent is restarted, or doing full sync e.g. after
recovery of the connectivity to rabbitmq, it is setting controller for
bridges in the openvswitch. That operation isn't idempotent, and even if
controller was already created, it will create new one. That, in some
cases can lead to short (1-2 seconds) traffic loss in the dataplane. We
observed that by doing UDP traffic test between 2 VMs:

- server A run:
sudo iperf3 -s -p 5000 -i 1

- server B run:
iperf3 -c 192.168.155.143 -p 5000 -u --length 1400 -b 1M -i 1 -t 3001

When neutron-ovs-agent on the node where server B runs is restarted,
packet loss can be observed in iperf, like:

[  5]  44.00-45.00  sec   122 KBytes   997 Kbits/sec  0.028 ms  0/89 (0%)  
iperf3: OUT OF ORDER - incoming packet = 4047 and received packet = 4048 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 4046 and received packet = 4049 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 4051 and received packet = 4053 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 4052 and received packet = 4054 AND SP = 5
[  5]  45.00-46.00  sec   123 KBytes  1.01 Mbits/sec  0.021 ms  4/90 (4.4%)  
[  5]  46.00-47.00  sec   122 KBytes   997 Kbits/sec  0.028 ms  0/89 (0%)  
iperf3: OUT OF ORDER - incoming packet = 4218 and received packet = 4219 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 4220 and received packet = 4221 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 4222 and received packet = 4223 AND SP = 5
[  5]  47.00-48.00  sec   122 KBytes   997 Kbits/sec  0.024 ms  3/89 (3.4%)

** Affects: neutron
     Importance: Medium
     Assignee: Slawek Kaplonski (slaweq)
         Status: Confirmed


** Tags: ovs

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1948642

Title:
  Configuration of the ovs controller by neutron-ovs-agent isn't
  idempotent

Status in neutron:
  Confirmed

Bug description:
  When neutron-ovs-agent is restarted, or doing full sync e.g. after
  recovery of the connectivity to rabbitmq, it is setting controller for
  bridges in the openvswitch. That operation isn't idempotent, and even
  if controller was already created, it will create new one. That, in
  some cases can lead to short (1-2 seconds) traffic loss in the
  dataplane. We observed that by doing UDP traffic test between 2 VMs:

  - server A run:
  sudo iperf3 -s -p 5000 -i 1

  - server B run:
  iperf3 -c 192.168.155.143 -p 5000 -u --length 1400 -b 1M -i 1 -t 3001

  When neutron-ovs-agent on the node where server B runs is restarted,
  packet loss can be observed in iperf, like:

  [  5]  44.00-45.00  sec   122 KBytes   997 Kbits/sec  0.028 ms  0/89 (0%)  
  iperf3: OUT OF ORDER - incoming packet = 4047 and received packet = 4048 AND SP = 5
  iperf3: OUT OF ORDER - incoming packet = 4046 and received packet = 4049 AND SP = 5
  iperf3: OUT OF ORDER - incoming packet = 4051 and received packet = 4053 AND SP = 5
  iperf3: OUT OF ORDER - incoming packet = 4052 and received packet = 4054 AND SP = 5
  [  5]  45.00-46.00  sec   123 KBytes  1.01 Mbits/sec  0.021 ms  4/90 (4.4%)  
  [  5]  46.00-47.00  sec   122 KBytes   997 Kbits/sec  0.028 ms  0/89 (0%)  
  iperf3: OUT OF ORDER - incoming packet = 4218 and received packet = 4219 AND SP = 5
  iperf3: OUT OF ORDER - incoming packet = 4220 and received packet = 4221 AND SP = 5
  iperf3: OUT OF ORDER - incoming packet = 4222 and received packet = 4223 AND SP = 5
  [  5]  47.00-48.00  sec   122 KBytes   997 Kbits/sec  0.024 ms  3/89 (3.4%)

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1948642/+subscriptions