yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1604370] [NEW] functional: test_legacy_router_ns_rebuild is unstable

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Jakub Libosvar <libosvar@xxxxxxxxxx>
Date: Tue, 19 Jul 2016 11:48:53 -0000
Reply-to: Bug 1604370 <1604370@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

logstash query: http://goo.gl/xfeSlj
13 hits in last 7 days

Example of failure: http://logs.openstack.org/01/312401/8/check/gate-
neutron-dsvm-functional/6aedd17/logs/dsvm-functional-
logs/neutron.tests.functional.agent.l3.test_legacy_router.L3AgentTestCase.test_legacy_router_ns_rebuild.txt.gz#_2016-07-19_10_45_28_248

Traceback:
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info [req-8e9eaf17-43d7-46e4-8179-84cb18fa106e - - - - -] Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-85bf65ee-a2"
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/common/utils.py", line 239, in call
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     return func(*args, **kwargs)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 1035, in process
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self.process_external(agent)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 817, in process_external
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self._process_external_gateway(ex_gw_port, agent.pd)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 695, in _process_external_gateway
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self.external_gateway_added(ex_gw_port, interface_name)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 660, in external_gateway_added
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     ex_gw_port, interface_name, self.ns_name, preserve_ips)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 606, in _external_gateway_added
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self._plug_external_gateway(ex_gw_port, interface_name, ns_name)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 573, in _plug_external_gateway
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     mtu=ex_gw_port.get('mtu'))
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/interface.py", line 251, in plug
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     bridge, namespace, prefix, mtu)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/interface.py", line 344, in plug_new
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     ns_dev.link.set_address(mac_address)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/ip_lib.py", line 496, in set_address
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self._as_root([], ('set', self.name, 'address', mac_address))
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/ip_lib.py", line 362, in _as_root
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     use_root_namespace=use_root_namespace)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/ip_lib.py", line 95, in _as_root
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     log_fail_as_error=self.log_fail_as_error)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/ip_lib.py", line 104, in _execute
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     log_fail_as_error=log_fail_as_error)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/utils.py", line 138, in execute
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     raise RuntimeError(msg)
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-85bf65ee-a2"
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info 
2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info

** Affects: neutron
     Importance: Undecided
         Status: New


** Tags: functional-tests

** Tags added: functional-tests

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1604370

Title:
  functional: test_legacy_router_ns_rebuild is unstable

Status in neutron:
  New

Bug description:
  logstash query: http://goo.gl/xfeSlj
  13 hits in last 7 days

  Example of failure: http://logs.openstack.org/01/312401/8/check/gate-
  neutron-dsvm-functional/6aedd17/logs/dsvm-functional-
  logs/neutron.tests.functional.agent.l3.test_legacy_router.L3AgentTestCase.test_legacy_router_ns_rebuild.txt.gz#_2016-07-19_10_45_28_248

  Traceback:
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info [req-8e9eaf17-43d7-46e4-8179-84cb18fa106e - - - - -] Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-85bf65ee-a2"
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/common/utils.py", line 239, in call
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     return func(*args, **kwargs)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 1035, in process
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self.process_external(agent)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 817, in process_external
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self._process_external_gateway(ex_gw_port, agent.pd)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 695, in _process_external_gateway
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self.external_gateway_added(ex_gw_port, interface_name)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 660, in external_gateway_added
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     ex_gw_port, interface_name, self.ns_name, preserve_ips)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 606, in _external_gateway_added
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self._plug_external_gateway(ex_gw_port, interface_name, ns_name)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/l3/router_info.py", line 573, in _plug_external_gateway
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     mtu=ex_gw_port.get('mtu'))
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/interface.py", line 251, in plug
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     bridge, namespace, prefix, mtu)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/interface.py", line 344, in plug_new
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     ns_dev.link.set_address(mac_address)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/ip_lib.py", line 496, in set_address
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     self._as_root([], ('set', self.name, 'address', mac_address))
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/ip_lib.py", line 362, in _as_root
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     use_root_namespace=use_root_namespace)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/ip_lib.py", line 95, in _as_root
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     log_fail_as_error=self.log_fail_as_error)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/ip_lib.py", line 104, in _execute
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     log_fail_as_error=log_fail_as_error)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info   File "neutron/agent/linux/utils.py", line 138, in execute
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info     raise RuntimeError(msg)
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-85bf65ee-a2"
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info 
  2016-07-19 10:45:28.248 27850 ERROR neutron.agent.l3.router_info

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1604370/+subscriptions
Follow ups

[Bug 1604370] Re: functional: test_legacy_router_ns_rebuild is unstable
From: OpenStack Infra, 2016-07-29