yahoo-eng-team team mailing list archive

[Slawek Kaplonski <1847203@xxxxxxxxxxxxxxxxxx>]

Thx Lars for details.
It seems for me that those routers are either not added to queue here:
https://github.com/openstack/neutron/blob/1c2e10f8595d2286bd9bec513bc5a346a84a6f7c/neutron/agent/l3/agent.py#L574
either not returned from each_update_to_next_resource() here:
https://github.com/openstack/neutron/blob/ad028b55cafd74c39b1eb0708dba45a6fcfff059/neutron/agent/common/resource_processing_queue.py#L171 - as routers returned from this method are processed in https://github.com/openstack/neutron/blob/1c2e10f8595d2286bd9bec513bc5a346a84a6f7c/neutron/agent/l3/agent.py#L671

Can You maybe add some additional debug messages on Your env to check
exactly where it is "stopped"?

** Changed in: neutron
       Status: Incomplete => Opinion

** Changed in: neutron
       Status: Opinion => New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1847203

Title:
  l3-agent stops processing router updates

Status in neutron:
  New

Bug description:
  In our work on upgrading from Queens to Rocky, we have stumbled upon
  some weird behaviour in neutron-l3-agent. After "a while" (usually
  ~days), the l3-agent will simply stop processing router updates. In
  the debug log, we see:

  Got routers updated notification :[u'1dea9d84-e5ec-44be-b37f-
  7f9070dd159e'] routers_updated /usr/lib/python2.7/dist-
  packages/neutron/agent/l3/agent.py:446

  But then nothing happens after that. We're testing this with adding
  and removing a floating IP.

  The problem is that we really have noe clue what happens, other than
  the observed symptoms, so we can't really provide a way to reproduce
  this..

  neutron-l3-agent     2:13.0.4-0ubuntu1~cloud0
  openvswitch-switch   2.10.0-0ubuntu2~cloud0

  Ubuntu 18.04 LTS, running the 4.15.0-51-generic kernel

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1847203/+subscriptions