← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #84442

[Bug 1892846] Re: AttributeError in l3-agent changing HA router to non-HA

  Thread Previous • Date Previous • • Thread Next 
Reviewed:  https://review.opendev.org/747922
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=489e0ead7297e3b17ca2bf8c4bea9701ad14a939
Submitter: Zuul
Branch:    master

commit 489e0ead7297e3b17ca2bf8c4bea9701ad14a939
Author: Slawek Kaplonski <skaplons@xxxxxxxxxx>
Date:   Tue Aug 25 14:44:26 2020 +0200

    Fix migration from the HA to non-HA routers
    
    In case if during switching HA router to be down, there will be any
    failure, router_info will be stored in L3 agent's cache as HaRouter.
    In case when next update on the router is migration to non-HA router
    this is wrong class and it causes other issues, e.g. with
    remove_vip_by_ip_address() which is correct only for HA routers.
    
    This patch fixes that issue by adding check of the router's ha and
    distributed flags and update local cache with new router_info class
    in case if at least one of those flags don't match.
    
    Change-Id: Ib0d3a501f88c149baea7d715c7cfe5811bc85e4f
    Closes-Bug: #1892846


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1892846

Title:
  AttributeError in l3-agent changing HA router to non-HA

Status in neutron:
  Fix Released

Bug description:
  In case when router is migrated from HA to non-HA one, it has to be disabled and then migrated.
  If that is done fast enough, and some error will occur in router_remove method (https://github.com/openstack/neutron/blob/master/neutron/agent/l3/agent.py#L547) old router_info will be added to the self.router_info dict.
  In such case it will be still HA router added to self.router_info but as this router is already switched to be non-HA, it will use wrong RouterInfo class and errors like:

  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info [-] 'NoneType' object has no attribute 'remove_vip_by_ip_address': AttributeError: 'NoneType' object has no attribute 'remove_vip_by_ip_address'
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info     return func(*args, **kwargs)
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1186, in process
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info     self._process_internal_ports()
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 594, in _process_internal_ports
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info     self.internal_network_added(p)
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 356, in internal_network_added
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info     port, self.get_internal_device_name, router.INTERNAL_DEV_PREFIX)
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 351, in _plug_ha_router_port
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info     self._disable_ipv6_addressing_on_interface(interface_name)
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 312, in _disable_ipv6_addressing_on_interface
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info     self._remove_vip(ipv6_lladdr)
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 236, in _remove_vip
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info     instance.remove_vip_by_ip_address(ip_cidr)
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info AttributeError: 'NoneType' object has no attribute 'remove_vip_by_ip_address'
  2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 8a8e034d-0c53-4d06-a936-9bb53b064f02: AttributeError: 'NoneType' object has no attribute 'remove_vip_by_ip_address'
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent Traceback (most recent call last):
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 694, in _process_routers_if_compatible
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     self._process_router_if_compatible(router)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 540, in _process_router_if_compatible
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     self._process_updated_router(router)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 581, in _process_updated_router
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     ri.process()
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 485, in process
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     super(HaRouter, self).process()
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 161, in call
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     self.logger(e)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     self.force_reraise()
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     six.reraise(self.type_, self.value, self.tb)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/six.py", line 675, in reraise
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     raise value
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     return func(*args, **kwargs)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1186, in process
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     self._process_internal_ports()
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 594, in _process_internal_ports
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     self.internal_network_added(p)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 356, in internal_network_added
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     port, self.get_internal_device_name, router.INTERNAL_DEV_PREFIX)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 351, in _plug_ha_router_port
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     self._disable_ipv6_addressing_on_interface(interface_name)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 312, in _disable_ipv6_addressing_on_interface
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     self._remove_vip(ipv6_lladdr)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent   File "/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 236, in _remove_vip
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent     instance.remove_vip_by_ip_address(ip_cidr)
  2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'remove_vip_by_ip_address'

  will occur in L3 agent.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1892846/+subscriptions

References

  Thread Previous • Date Previous • • Thread Next