yahoo-eng-team team mailing list archive

[OpenStack Infra <1817435@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/640006
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fe4e7724cde61a036262a392abd1be76013f699c
Submitter: Zuul
Branch:    master

commit fe4e7724cde61a036262a392abd1be76013f699c
Author: Slawek Kaplonski <skaplons@xxxxxxxxxx>
Date:   Thu Feb 28 12:32:18 2019 +0100

    Don't disable radvd if radvd if it wasn't initialized
    
    In some cases on dvr ha router it may happend that
    RouterInfo.radvd.disable() will be called even if
    radvd DaemonMonitor wasn't initialized earlier and it is
    None.
    To prevent exception in such case, this patch adds check
    if DaemonMonitor is not None to call disable() method on
    it.
    
    Change-Id: Ib9b5f4eeae6e4cebcb958928e6521cf1d69b049c
    Closes-Bug: #1817435


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1817435

Title:
  Error while stoping radvd on dvr ha router

Status in neutron:
  Fix Released

Bug description:
  In neutron-tempest-dvr-multinode-ha-full I found errors in L3 agent's logs. Error example is on:
  http://logs.openstack.org/79/633979/11/check/neutron-tempest-dvr-ha-multinode-full/415f056/controller/logs/screen-q-l3.txt.gz?level=ERROR#_Feb_22_12_00_34_232612

  Error looks like:

  Feb 22 12:00:34.219810 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent Exception: Unable to process HA router 1f90a750-d3af-454a-9031-29f399cf6457 without HA port
  Feb 22 12:00:34.219810 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent 
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent [-] Error while deleting router 1f90a750-d3af-454a-9031-29f399cf6457: AttributeError: 'NoneType' object has no attribute 'disable'
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent Traceback (most recent call last):
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 447, in _router_added
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     ri.delete()
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/ha_router.py", line 461, in delete
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     super(HaRouter, self).delete()
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 431, in delete
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     self.disable_radvd()
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 547, in disable_radvd
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     self.radvd.disable()
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'disable'
  Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent 
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 1f90a750-d3af-454a-9031-29f399cf6457: Exception: Unable to process HA router 1f90a750-d3af-454a-9031-29f399cf6457 without HA port
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent Traceback (most recent call last):
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 687, in _process_routers_if_compatible
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     self._process_router_if_compatible(router)
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 562, in _process_router_if_compatible
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     self._process_added_router(router)
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 567, in _process_added_router
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     self._router_added(router['id'], router)
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 450, in _router_added
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     router_id)
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     self.force_reraise()
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     six.reraise(self.type_, self.value, self.tb)
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 439, in _router_added
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     ri.initialize(self.process_monitor)
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent   File "/opt/stack/neutron/neutron/agent/l3/ha_router.py", line 126, in initialize
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent     raise Exception(msg)
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent Exception: Unable to process HA router 1f90a750-d3af-454a-9031-29f399cf6457 without HA port
  Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent 

  
  Funny thing is that I found it on my patch https://review.openstack.org/#/c/633979/ which is migrating job definition to zuulv3 syntax. There is no such error in existing legacy job because it is not creates HA routers by default. So this is another issue that neutron-tempest-dvr-HA-multinode-full job is actually not HA but that can be fixed in https://review.openstack.org/#/c/633979/ after this issue reported here will be fixed.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1817435/+subscriptions