yahoo-eng-team team mailing list archive

[Brian Haley <1510757@xxxxxxxxxxxxxxxxxx>]

Closing as I don't think this happens any more.

** Changed in: neutron
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1510757

Title:
  tempest test api.network;l3 agent can't delete HA-router

Status in neutron:
  Invalid

Bug description:
  I use tempest test my company's openstack environment.

  # tox -eall -- tempest.api.network

  When tox finished. Log in L3-agent always show:

  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent [-] Error while deleting router da4b28ce-33b1-4000-8609-a41a2ab8c982
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent Traceback (most recent call last):
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 344, in _safe_router_removed
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     self._router_removed(router_id)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 362, in _router_removed
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     ri.delete(self)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 364, in delete
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     super(HaRouter, self).delete(agent)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 273, in delete
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     self.process(agent)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 370, in process
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     super(HaRouter, self).process(agent)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/common/utils.py", line 359, in call
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     self.logger(e)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in __exit__
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     six.reraise(self.type_, self.value, self.tb)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/common/utils.py", line 356, in call
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     return func(*args, **kwargs)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 695, in process
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     self.routes_updated()
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 181, in routes_updated
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     instance = self._get_keepalived_instance()
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent   File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 131, in _get_keepalived_instance
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent     return self.keepalived_manager.config.get_instance(self.ha_vr_id)
  2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'config'

  I think the reason is tempest create and delete router too fast.
  when l3-agent create ha-router,tempest delete the router,and neutron-server delete ha-interface

  https://github.com/openstack/neutron/blob/master/neutron/agent/l3/ha_router.py#L79

  class HaRouter can't initialized without ha-interface information,it just return without _init_keepalived_manager
  When l3-agent delete router,it report AttributeError: 'NoneType'.

  When l3 agent can't delete the router,l3 agent always fullsync with neutron-server every 30 seconds.
  In controller, the neutron-server cpu always 70%..^-^

  L3 agent should add a check before create HArouter.If it find ha-
  interface is none ,means router has been deleted in neutron-server.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1510757/+subscriptions