yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1609248] [NEW] HA: ha router failed to process floating IP

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: LIU Yulong <yulong@xxxxxxxxxxxxx>
Date: Wed, 03 Aug 2016 06:39:55 -0000
Reply-to: Bug 1609248 <1609248@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

ENV:
stable/mitaka

How to reproduce:
1. create a HA router
neutron router-create --ha True --distributed False test1

2. set the HA router gateway with more than one IP
neutron router-gateway-set --fixed-ip ip_address=172.16.5.110 --fixed-ip ip_address=172.16.5.111 test1 public

3. reset (update) the HA router  gateway IP with a lower IP address
neutron router-gateway-set --fixed-ip ip_address=172.16.5.109 --fixed-ip ip_address=172.16.5.110 test1 public

172.16.5.109 is lower that 172.16.5.110, then l3 agent will get the
following trace:


2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info [-] L3 agent failure to setup floating IPs2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 402, in configure_fip_addresses2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     return self.process_floating_ip_addresses(interface_name)2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 396, in process_floating_ip_addresses
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     self.remove_floating_ip(device, ip_cidr)
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 271, in remove_floating_ip
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     super(HaRouter, self).remove_floating_ip(device, ip_cidr) 
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 306, in remove_floating_ip
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     device.delete_addr_and_conntrack_state(ip_cidr) 
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 290, in delete_addr_and_conntrack_state
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     self.addr.delete(cidr)
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 580, in delete
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     'dev', self.name))2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 361, in _as_root2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     use_root_namespace=use_root_namespace)2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 94, in _as_root2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     log_fail_as_error=self.log_fail_as_error)
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 103, in _execute
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     log_fail_as_error=log_fail_as_error)
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 140, in execute
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     raise RuntimeError(msg)
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info RuntimeError: Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: Cannot assign requested address
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info
2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info
2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info [-] Failed to process floating IPs.
2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 965, in process_external
2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info     fip_statuses = self.configure_fip_addresses(interface_name)
2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 407, in configure_fip_addresses
2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info     raise n_exc.FloatingIpSetupException(msg)
2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info FloatingIpSetupException: L3 agent failure to setup floating IPs
2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1609248

Title:
  HA: ha router failed to process floating IP

Status in neutron:
  New

Bug description:
  ENV:
  stable/mitaka

  How to reproduce:
  1. create a HA router
  neutron router-create --ha True --distributed False test1

  2. set the HA router gateway with more than one IP
  neutron router-gateway-set --fixed-ip ip_address=172.16.5.110 --fixed-ip ip_address=172.16.5.111 test1 public

  3. reset (update) the HA router  gateway IP with a lower IP address
  neutron router-gateway-set --fixed-ip ip_address=172.16.5.109 --fixed-ip ip_address=172.16.5.110 test1 public

  172.16.5.109 is lower that 172.16.5.110, then l3 agent will get the
  following trace:

  
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info [-] L3 agent failure to setup floating IPs2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 402, in configure_fip_addresses2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     return self.process_floating_ip_addresses(interface_name)2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 396, in process_floating_ip_addresses
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     self.remove_floating_ip(device, ip_cidr)
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 271, in remove_floating_ip
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     super(HaRouter, self).remove_floating_ip(device, ip_cidr) 
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 306, in remove_floating_ip
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     device.delete_addr_and_conntrack_state(ip_cidr) 
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 290, in delete_addr_and_conntrack_state
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     self.addr.delete(cidr)
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 580, in delete
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     'dev', self.name))2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 361, in _as_root2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     use_root_namespace=use_root_namespace)2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 94, in _as_root2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     log_fail_as_error=self.log_fail_as_error)
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 103, in _execute
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     log_fail_as_error=log_fail_as_error)
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 140, in execute
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info     raise RuntimeError(msg)
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info RuntimeError: Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: Cannot assign requested address
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info
  2016-08-03 14:36:46.200 300 ERROR neutron.agent.l3.router_info
  2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info [-] Failed to process floating IPs.
  2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
  2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 965, in process_external
  2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info     fip_statuses = self.configure_fip_addresses(interface_name)
  2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 407, in configure_fip_addresses
  2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info     raise n_exc.FloatingIpSetupException(msg)
  2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info FloatingIpSetupException: L3 agent failure to setup floating IPs
  2016-08-03 14:36:46.201 300 ERROR neutron.agent.l3.router_info

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1609248/+subscriptions
Follow ups

[Bug 1609248] Re: HA: ha router failed to process floating IP
From: OpenStack Infra, 2016-08-20