yahoo-eng-team team mailing list archive

[OpenStack Infra <1606844@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/348372
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=31a7feea6b60dac138b00652d2f16982a3b25f78
Submitter: Jenkins
Branch:    master

commit 31a7feea6b60dac138b00652d2f16982a3b25f78
Author: Oleg Bondarev <obondarev@xxxxxxxxxxxx>
Date:   Thu Jul 28 17:03:22 2016 +0300

    L3 agent: check router namespace existence before delete
    
    Router namespace absence may lead to infinite loop in l3 agent trying
    to delete the router.
    This patch adds checks before going into namespace to prevent RuntimeError
    and following infinite loop.
    
    Closes-Bug: #1606844
    Change-Id: Iae95ccb8eeb06d0fd5fc7d71e63408b3f843b371


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1606844

Title:
  L3 agent constantly resyncing deleted router

Status in neutron:
  Fix Released

Bug description:
  No need to constantly resync router which was deleted and for which
  there is no namespace.

  Observed: l3 agent log full of

  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent [-] Error while deleting router 81ef46de-f7f9-4c5e-b787-c935e0af253a
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent Traceback (most recent call last):
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 359, in _safe_router_removed
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     self._router_removed(router_id)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 377, in _router_removed
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     ri.delete(self)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 347, in delete
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     self.process_delete(agent)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 385, in call
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     self.logger(e)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     self.force_reraise()
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     six.reraise(self.type_, self.value, self.tb)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 382, in call
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     return func(*args, **kwargs)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 947, in process_delete
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     self._process_internal_ports(agent.pd)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 530, in _process_internal_ports
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     existing_devices = self._get_existing_devices()
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 413, in _get_existing_devices
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     ip_devs = ip_wrapper.get_devices(exclude_loopback=True)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 130, in get_devices
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     log_fail_as_error=self.log_fail_as_error
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 140, in execute
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent     raise RuntimeError(msg)
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "qrouter-81ef46de-f7f9-4c5e-b787-c935e0af253a": No such file or directory
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent
  2016-07-26 14:00:45.224 13360 ERROR neutron.agent.l3.agent
  2016-07-26 14:00:45.236 13360 ERROR neutron.agent.linux.utils [-] Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "qrouter-81ef46de-f7f9-4c5e-b787-c935e0af253a": No such file or directory

  this consumes memory, cpu, disk.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1606844/+subscriptions