yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1817306] [NEW] Failed gARP for floating IP in l3 agent logs

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Oleg Bondarev <obondarev@xxxxxxxxxxxx>
Date: Fri, 22 Feb 2019 11:43:11 -0000
Reply-to: Bug 1817306 <1817306@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

release: Pike.

When a DVR router with centralized floating IPs is rescheduled from down l3 agent to another,
following traces are seen on the destination agent:

2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib [-] Failed sending gratuitous ARP to 10.13.250.14 on qg-af0de258-a8 in namespace snat-afe70a67-a007-4bcf-93ac-099aad63411c: Exit code: 2; Stdin: ; Stdout: ; Stderr: bind: Cannot assign requested address
: ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: bind: Cannot assign requested address
2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib Traceback (most recent call last):
2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 1097, in _arping
2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib     ip_wrapper.netns.execute(arping_cmd, extra_ok_codes=[1])
2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 916, in execute
2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib     log_fail_as_error=log_fail_as_error, **kwargs)
2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 151, in execute
2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib     raise ProcessExecutionError(msg, returncode=returncode)
2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: bind: Cannot assign requested address

Earlier in logs following is seen:

2019-02-22 07:52:14,894.894 9528 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'snat-afe70a67-a007-4bcf-93ac-099aad63411c', 'ip', '-4', 'addr', 'del', '10.13.250.14/32', 'dev', 'qg-af0de258-a8'] execute_rootwrap_daemon /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:108
2019-02-22 07:52:14,922.922 9528 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'snat-afe70a67-a007-4bcf-93ac-099aad63411c', 'conntrack', '-D', '-d', '10.13.250.14'] execute_rootwrap_daemon /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:108

So centralized floating ip is deleted from gw device for some reason.

** Affects: neutron
     Importance: Undecided
     Assignee: Oleg Bondarev (obondarev)
         Status: New


** Tags: l3-dvr-backlog

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1817306

Title:
  Failed gARP for floating IP in l3 agent logs

Status in neutron:
  New

Bug description:
  release: Pike.

  When a DVR router with centralized floating IPs is rescheduled from down l3 agent to another,
  following traces are seen on the destination agent:

  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib [-] Failed sending gratuitous ARP to 10.13.250.14 on qg-af0de258-a8 in namespace snat-afe70a67-a007-4bcf-93ac-099aad63411c: Exit code: 2; Stdin: ; Stdout: ; Stderr: bind: Cannot assign requested address
  : ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: bind: Cannot assign requested address
  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib Traceback (most recent call last):
  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 1097, in _arping
  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib     ip_wrapper.netns.execute(arping_cmd, extra_ok_codes=[1])
  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 916, in execute
  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib     log_fail_as_error=log_fail_as_error, **kwargs)
  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 151, in execute
  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib     raise ProcessExecutionError(msg, returncode=returncode)
  2019-02-21 14:55:59,150.150 24730 ERROR neutron.agent.linux.ip_lib ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: bind: Cannot assign requested address

  Earlier in logs following is seen:

  2019-02-22 07:52:14,894.894 9528 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'snat-afe70a67-a007-4bcf-93ac-099aad63411c', 'ip', '-4', 'addr', 'del', '10.13.250.14/32', 'dev', 'qg-af0de258-a8'] execute_rootwrap_daemon /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:108
  2019-02-22 07:52:14,922.922 9528 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'snat-afe70a67-a007-4bcf-93ac-099aad63411c', 'conntrack', '-D', '-d', '10.13.250.14'] execute_rootwrap_daemon /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:108

  So centralized floating ip is deleted from gw device for some reason.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1817306/+subscriptions
Follow ups

[Bug 1817306] Re: Failed gARP for floating IP in l3 agent logs
From: OpenStack Infra, 2019-03-03