yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #70645
[Bug 1741889] Re: functional: DbAddCommand sometimes times out after 10 seconds
Reviewed: https://review.openstack.org/537241
Committed: https://git.openstack.org/cgit/openstack/ovsdbapp/commit/?id=1810faecc9ad2345f3e2f9185ac64194c5a0d711
Submitter: Zuul
Branch: master
commit 1810faecc9ad2345f3e2f9185ac64194c5a0d711
Author: Terry Wilson <twilson@xxxxxxxxxx>
Date: Wed Jan 24 03:41:13 2018 +0000
Ensure idl.run() called on TRY_AGAIN
If the DB connection drops after Connection.run() calls idl.run()
but before Transaction.do_commit(), then it is possible for
idl.commit() to return TRY_AGAIN due to the failure of session.send().
This prevents commit_block() from running idl.run() which would
actually trigger the DB reconnect, causing each successive call to
commit_block() to fail in the same way until we stop trying again
due to timeout and Connection.run() calls idl.run() again.
For now, we can just ensure that idl.run() is called when we get a
TRY_AGAIN. It might be worth it to refactor this a bit to not even
use commit_block().
Change-Id: I77402bde3fc4d45c770b9f48882870cfc91d719d
Closes-Bug: #1741889
** Changed in: ovsdbapp
Status: New => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1741889
Title:
functional: DbAddCommand sometimes times out after 10 seconds
Status in neutron:
New
Status in ovsdbapp:
Fix Released
Bug description:
Traceback (most recent call last):
File "/opt/stack/new/neutron/.tox/dsvm-functional/local/lib/python2.7/site-packages/fixtures/fixture.py", line 197, in setUp
self._setUp()
File "neutron/tests/common/net_helpers.py", line 721, in _setUp
self.bridge = common_base.create_resource(self.prefix, ovs.add_bridge)
File "neutron/tests/common/base.py", line 47, in create_resource
return creation_func(name, *args, **kwargs)
File "neutron/agent/common/ovs_lib.py", line 140, in add_bridge
br.create()
File "neutron/agent/common/ovs_lib.py", line 255, in create
FAILMODE_SECURE))
File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
File "/opt/stack/new/neutron/.tox/dsvm-functional/local/lib/python2.7/site-packages/ovsdbapp/api.py", line 94, in transaction
self._nested_txn = None
File "/opt/stack/new/neutron/.tox/dsvm-functional/local/lib/python2.7/site-packages/ovsdbapp/api.py", line 54, in __exit__
self.result = self.commit()
File "/opt/stack/new/neutron/.tox/dsvm-functional/local/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 57, in commit
timeout=self.timeout)
ovsdbapp.exceptions.TimeoutException: Commands [<ovsdbapp.schema.open_vswitch.commands.AddBridgeCommand object at 0x7fdcc274a9d0>, <ovsdbapp.schema.open_vswitch.commands.DbAddCommand object at 0x7fdcc274aad0>] exceeded timeout 10 seconds
logstash query: http://logstash.openstack.org/#dashboard/file/logstash.json?query=build_name%3A%5C%22legacy-neutron-dsvm-functional%5C%22%20AND%20message%3A%5C%22ovsdbapp.exceptions.TimeoutException%3A%20Commands%5C%22%20AND%20tags%3Aconsole%20AND%20build_status%3AFAILURE
In the logs we can see AddBridgeCommand result returning TRY_AGAIN over and over again:
http://logs.openstack.org/70/531070/4/check/legacy-neutron-dsvm-functional/9aa7107/logs/dsvm-functional-logs/neutron.tests.functional.agent.test_dhcp_agent.DHCPAgentOVSTestCase.test_agent_mtu_set_on_interface_driver.txt.gz#_2018-01-08_04_20_06_754
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1741889/+subscriptions
References