yahoo-eng-team team mailing list archive

[Edward Hope-Morley <1920975@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Neutron DVR uses proxy_arp in fip namespaces to respond to arp requests
for instance floating ips. In doing so it is susceptible to a random
delay up to by default 800ms which is added to the time taken to respond
to an arp request that has to be proxied i.e.

# ip netns exec fip-a297543b-9ef9-4bd5-b1ca-e85a726c1726 sysctl net.ipv4.{conf.fg-51f3e07b-2d.proxy_arp,neigh.fg-51f3e07b-2d.proxy_delay}
net.ipv4.conf.fg-51f3e07b-2d.proxy_arp = 1
net.ipv4.neigh.fg-51f3e07b-2d.proxy_delay = 80

The result of this is seen when e.g. you ping a vm fip and the first
request takes significantly longer than subsequent requests:

$ ping -c 5 10.5.150.90
PING 10.5.150.90 (10.5.150.90) 56(84) bytes of data.
64 bytes from 10.5.150.90: icmp_seq=1 ttl=60 time=491 ms
64 bytes from 10.5.150.90: icmp_seq=2 ttl=60 time=1.08 ms
64 bytes from 10.5.150.90: icmp_seq=3 ttl=60 time=1.39 ms
64 bytes from 10.5.150.90: icmp_seq=4 ttl=60 time=1.16 ms
64 bytes from 10.5.150.90: icmp_seq=5 ttl=60 time=1.03 ms

--- 10.5.150.90 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4007ms
rtt min/avg/max/mdev = 1.034/99.157/491.134/195.988 ms

To repro again simply delete arp entry for fip from fip ns of source
compute host.

By kernel standards this behaviour is by-design when using the default
settings but some workloads may be impacted by this initial delay
especially e.g. in loaded environments where the arp caches are under
strain and hitting gc_thresh limits.

** Affects: charm-neutron-openvswitch
     Importance: Undecided
         Status: New

** Affects: neutron
     Importance: Undecided
         Status: New

** Summary changed:

- neutron dvr should lower arp_delay when using arp_proxy
+ neutron dvr should lower proxy_delay when using arp_proxy

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1920975

Title:
  neutron dvr should lower proxy_delay when using proxy_arp

Status in OpenStack neutron-openvswitch charm:
  New
Status in neutron:
  New

Bug description:
  Neutron DVR uses proxy_arp in fip namespaces to respond to arp
  requests for instance floating ips. In doing so it is susceptible to a
  random delay up to by default 800ms which is added to the time taken
  to respond to an arp request that has to be proxied i.e.

  # ip netns exec fip-a297543b-9ef9-4bd5-b1ca-e85a726c1726 sysctl net.ipv4.{conf.fg-51f3e07b-2d.proxy_arp,neigh.fg-51f3e07b-2d.proxy_delay}
  net.ipv4.conf.fg-51f3e07b-2d.proxy_arp = 1
  net.ipv4.neigh.fg-51f3e07b-2d.proxy_delay = 80

  The result of this is seen when e.g. you ping a vm fip and the first
  request takes significantly longer than subsequent requests:

  $ ping -c 5 10.5.150.90
  PING 10.5.150.90 (10.5.150.90) 56(84) bytes of data.
  64 bytes from 10.5.150.90: icmp_seq=1 ttl=60 time=491 ms
  64 bytes from 10.5.150.90: icmp_seq=2 ttl=60 time=1.08 ms
  64 bytes from 10.5.150.90: icmp_seq=3 ttl=60 time=1.39 ms
  64 bytes from 10.5.150.90: icmp_seq=4 ttl=60 time=1.16 ms
  64 bytes from 10.5.150.90: icmp_seq=5 ttl=60 time=1.03 ms

  --- 10.5.150.90 ping statistics ---
  5 packets transmitted, 5 received, 0% packet loss, time 4007ms
  rtt min/avg/max/mdev = 1.034/99.157/491.134/195.988 ms

  To repro again simply delete arp entry for fip from fip ns of source
  compute host.

  By kernel standards this behaviour is by-design when using the default
  settings but some workloads may be impacted by this initial delay
  especially e.g. in loaded environments where the arp caches are under
  strain and hitting gc_thresh limits.

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-openvswitch/+bug/1920975/+subscriptions