yahoo-eng-team team mailing list archive

[Swaminathan Vasudevan <swaminathan.vasudevan@xxxxxx>]

Public bug reported:

The ARP entries are dropped by DVR routers when the 'qr' device does not
exist in the namespace.

There are two ways in the L3 agent the ARP entries are updated.
Once when an internal csnat port is created, then arp entries added from the 'dvr_local_router' by calling the "set_subnet_arp_info" which in turn calls the "_update_arp_entry".

There is another time, when an arp update "rpc" message comes from the
Server to the agent as "add_arp_entry" or "delete_arp_entry" which
inturn calls "_update_arp_entry".

We have seen log traces that shows that the arp update message comes
before the "qr" device is ready. So we get to drop those arp message.

We need to kind of cache those arp messages and update the router-
namespace when the "qr" device is ready.

If you see the message below, we are checking for the device and
throwing a warning message that the device is not ready, but the arp
entries are not saved anywere. They are dropped.

2015-09-24 18:45:30.150 WARNING neutron.agent.l3.dvr_local_router [req-
0565ce3a-905d-43fa-a6f3-1a07df6c6c2b None None] Arp operation add failed
for device qr-b672ffde-cd, since the device does not exist anymore. The
device might have been concurrently deleted or not created yet.

If you see here the internal_network 'qr' device is added later.

2015-09-24 18:45:30.367 DEBUG neutron.agent.l3.router_info [req-
7e5722e4-5fef-4889-9372-8cf1218522a2 None None] adding internal network:
prefix(qr-), port(b672ffde-cd80-49eb-9817-58436fa8e8fd)
_internal_network_added
/opt/stack/new/neutron/neutron/agent/l3/router_info.py:300

** Affects: neutron
     Importance: Undecided
     Assignee: Swaminathan Vasudevan (swaminathan-vasudevan)
         Status: In Progress


** Tags: l3-dvr-backlog

** Changed in: neutron
       Status: New => Confirmed

** Changed in: neutron
     Assignee: (unassigned) => Swaminathan Vasudevan (swaminathan-vasudevan)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1501086

Title:
  ARP entries dropped by DVR routers when the qr device is not ready or
  present

Status in neutron:
  In Progress

Bug description:
  The ARP entries are dropped by DVR routers when the 'qr' device does
  not exist in the namespace.

  There are two ways in the L3 agent the ARP entries are updated.
  Once when an internal csnat port is created, then arp entries added from the 'dvr_local_router' by calling the "set_subnet_arp_info" which in turn calls the "_update_arp_entry".

  There is another time, when an arp update "rpc" message comes from the
  Server to the agent as "add_arp_entry" or "delete_arp_entry" which
  inturn calls "_update_arp_entry".

  We have seen log traces that shows that the arp update message comes
  before the "qr" device is ready. So we get to drop those arp message.

  We need to kind of cache those arp messages and update the router-
  namespace when the "qr" device is ready.

  If you see the message below, we are checking for the device and
  throwing a warning message that the device is not ready, but the arp
  entries are not saved anywere. They are dropped.

  2015-09-24 18:45:30.150 WARNING neutron.agent.l3.dvr_local_router
  [req-0565ce3a-905d-43fa-a6f3-1a07df6c6c2b None None] Arp operation add
  failed for device qr-b672ffde-cd, since the device does not exist
  anymore. The device might have been concurrently deleted or not
  created yet.

  If you see here the internal_network 'qr' device is added later.

  2015-09-24 18:45:30.367 DEBUG neutron.agent.l3.router_info [req-
  7e5722e4-5fef-4889-9372-8cf1218522a2 None None] adding internal
  network: prefix(qr-), port(b672ffde-cd80-49eb-9817-58436fa8e8fd)
  _internal_network_added
  /opt/stack/new/neutron/neutron/agent/l3/router_info.py:300

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1501086/+subscriptions