yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1892362] Re: Restarting L3 agent when PD is used fails due to IPAddressAlreadyExists error

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: OpenStack Infra <1892362@xxxxxxxxxxxxxxxxxx>
Date: Tue, 25 Aug 2020 09:07:59 -0000
Reply-to: Bug 1892362 <1892362@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Reviewed:  https://review.opendev.org/747393
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=81d375d39ab1affe4b0a29437eaabc387fb1c570
Submitter: Zuul
Branch:    master

commit 81d375d39ab1affe4b0a29437eaabc387fb1c570
Author: Slawek Kaplonski <skaplons@xxxxxxxxxx>
Date:   Fri Aug 21 13:15:34 2020 +0200

    Handle properly existing LLA address during l3 agent restart
    
    In case when L3 agent is hosting routers which have got subnets
    with Prefix Delegation enabled, agent couldn't properly handle
    IpAddressAlreadyExists exception raised when pd module tries to
    configure link local IPv6 addresses.
    
    Now this is fixed and L3 agent can restart without problems in such
    case.
    
    Change-Id: Icc995f7b2b465921e41342711d17539f16ead0ce
    Closes-Bug: #1892362


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1892362

Title:
  Restarting L3 agent when PD is used fails due to
  IPAddressAlreadyExists error

Status in neutron:
  Fix Released

Bug description:
  When there is router with subnet with enabled prefix delegation
  plugged and L3 agent is restarted it's failing with error like:

  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 7ef0983f-60a4-4389-909c-785eb6c9a51e: neutron.privileged.agent.linux.ip_lib.IpAddressAlreadyExists: IP address fe80::f816:3eff:fec8:b9a8 already configured on qg-351b89d8-61.
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent Traceback (most recent call last):
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 694, in _process_routers_if_compatible
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     self._process_router_if_compatible(router)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 540, in _process_router_if_compatible
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     self._process_updated_router(router)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 581, in _process_updated_router
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     ri.process()
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 161, in call
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     self.logger(e)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     self.force_reraise()
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     six.reraise(self.type_, self.value, self.tb)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     raise value
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     return func(*args, **kwargs)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1191, in process
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     self.process_external()
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 949, in process_external
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     self._process_external_gateway(ex_gw_port)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 833, in _process_external_gateway
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     interface_name)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 328, in inner
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     return f(*args, **kwargs)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/linux/pd.py", line 144, in add_gw_interface
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     self._add_lla(router, bind_lla_with_mask)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/linux/pd.py", line 220, in _add_lla
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     'link')
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/linux/interface.py", line 208, in add_ipv6_addr
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     device.addr.add(str(net), scope)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 508, in add
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     add_broadcast)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 872, in add_ip_address
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     device, namespace, scope, broadcast)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py", line 50, in sync_inner
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     return input_func(*args, **kwargs)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 244, in _wrap
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     return self.channel.remote_call(name, args, kwargs)
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 203, in remote_call
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent     raise exc_type(*result[2])
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent neutron.privileged.agent.linux.ip_lib.IpAddressAlreadyExists: IP address fe80::f816:3eff:fec8:b9a8 already configured on qg-351b89d8-61.
  2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent

  To workaround this problem I had to e.g. remove qrouter-XXX namespace
  from the node and then restart L3 agent. In that case all was
  configured without this error.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1892362/+subscriptions
References

[Bug 1892362] [NEW] Restarting L3 agent when PD is used fails due to IPAddressAlreadyExists error
From: Slawek Kaplonski, 2020-08-20