yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1640202] Re: L3 agent failure to setup floating IPs

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Rodolfo Alonso <1640202@xxxxxxxxxxxxxxxxxx>
Date: Fri, 16 Dec 2022 16:10:45 -0000
Reply-to: Bug 1640202 <1640202@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Bug closed due to lack of activity, please feel free to reopen if
needed.

** Changed in: neutron
       Status: Confirmed => Won't Fix

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1640202

Title:
  L3 agent failure to setup floating IPs

Status in neutron:
  Won't Fix

Bug description:
  Hello,

  We have a Mitaka deploy with several projects running, with 3
  controller nodes and 4 compute nodes. After restarting the L3 agent,
  for one specific tenant the floating IPs that are attached to an
  instance are not actually attached, L3 agent is throwing the following
  error:

  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info [-] L3 agent failure to setup floating IPs
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 342, in configure_fip_addresses
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     return self.process_floating_ip_addresses(interface_name)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 314, in process_floating_ip_addresses
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     fip, interface_name, device)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/dvr_local_router.py", line 157, in add_floating_ip
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     self.floating_ip_added_dist(fip, ip_cidr)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/dvr_local_router.py", line 88, in floating_ip_added_dist
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     device.route.add_route(fip_cidr, str(rtr_2_fip.ip))
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 841, in add_route
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     self._run_as_root_detect_device_not_found([ip_version], tuple(args))
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 699, in _run_as_root_detect_device_not_found
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     raise exceptions.DeviceNotFoundError(device_name=self.name)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     self.force_reraise()
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     six.reraise(self.type_, self.value, self.tb)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 694, in _run_as_root_detect_device_not_found
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     return self._as_root(*args, **kwargs)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 361, in _as_root
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     use_root_namespace=use_root_namespace)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 94, in _as_root
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     log_fail_as_error=self.log_fail_as_error)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 103, in _execute
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     log_fail_as_error=log_fail_as_error)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 140, in execute
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info     raise RuntimeError(msg)
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info RuntimeError: Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: Network is unreachable
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info 
  2016-11-08 08:56:02.061 12466 ERROR neutron.agent.l3.router_info 
  2016-11-08 08:56:02.063 12466 ERROR neutron.agent.l3.router_info [-] Failed to process floating IPs.
  2016-11-08 08:56:02.063 12466 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
  2016-11-08 08:56:02.063 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 815, in process_external
  2016-11-08 08:56:02.063 12466 ERROR neutron.agent.l3.router_info     fip_statuses = self.configure_fip_addresses(interface_name)
  2016-11-08 08:56:02.063 12466 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 347, in configure_fip_addresses
  2016-11-08 08:56:02.063 12466 ERROR neutron.agent.l3.router_info     raise n_exc.FloatingIpSetupException(msg)
  2016-11-08 08:56:02.063 12466 ERROR neutron.agent.l3.router_info FloatingIpSetupException: L3 agent failure to setup floating IPs
  2016-11-08 08:56:02.063 12466 ERROR neutron.agent.l3.router_info 

  As of now, every time we try to attach/detach the floating IP on an
  instance the L3 agent shows the same error. Neutron package versions
  are:

  ii  neutron-common                              2:8.2.0-0ubuntu1~cloud0
  ii  neutron-dhcp-agent                          2:8.2.0-0ubuntu1~cloud0
  ii  neutron-l3-agent                            2:8.2.0-0ubuntu1~cloud0
  ii  neutron-metadata-agent                      2:8.2.0-0ubuntu1~cloud0
  ii  neutron-openvswitch-agent                   2:8.2.0-0ubuntu1~cloud0
  ii  neutron-plugin-ml2                          2:8.2.0-0ubuntu1~cloud0

  OS: Ubuntu 14.04, uname: Linux compute01 3.16.0-77-generic
  #99~14.04.1-Ubuntu SMP Tue Jun 28 19:17:10 UTC 2016 x86_64 x86_64
  x86_64 GNU/Linux

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1640202/+subscriptions
References

[Bug 1640202] [NEW] L3 agent failure to setup floating IPs
From: Nicolas Vila, 2016-11-08