yahoo-eng-team team mailing list archive

[OpenStack Infra <1741889@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/537241
Committed: https://git.openstack.org/cgit/openstack/ovsdbapp/commit/?id=1810faecc9ad2345f3e2f9185ac64194c5a0d711
Submitter: Zuul
Branch:    master

commit 1810faecc9ad2345f3e2f9185ac64194c5a0d711
Author: Terry Wilson <twilson@xxxxxxxxxx>
Date:   Wed Jan 24 03:41:13 2018 +0000

    Ensure idl.run() called on TRY_AGAIN
    
    If the DB connection drops after Connection.run() calls idl.run()
    but before Transaction.do_commit(), then it is possible for
    idl.commit() to return TRY_AGAIN due to the failure of session.send().
    This prevents commit_block() from running idl.run() which would
    actually trigger the DB reconnect, causing each successive call to
    commit_block() to fail in the same way until we stop trying again
    due to timeout and Connection.run() calls idl.run() again.
    
    For now, we can just ensure that idl.run() is called when we get a
    TRY_AGAIN. It might be worth it to refactor this a bit to not even
    use commit_block().
    
    Change-Id: I77402bde3fc4d45c770b9f48882870cfc91d719d
    Closes-Bug: #1741889


** Changed in: ovsdbapp
       Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1741889

Title:
  functional: DbAddCommand sometimes times out after 10 seconds

Status in neutron:
  New
Status in ovsdbapp:
  Fix Released

Bug description:
  Traceback (most recent call last):
    File "/opt/stack/new/neutron/.tox/dsvm-functional/local/lib/python2.7/site-packages/fixtures/fixture.py", line 197, in setUp
      self._setUp()
    File "neutron/tests/common/net_helpers.py", line 721, in _setUp
      self.bridge = common_base.create_resource(self.prefix, ovs.add_bridge)
    File "neutron/tests/common/base.py", line 47, in create_resource
      return creation_func(name, *args, **kwargs)
    File "neutron/agent/common/ovs_lib.py", line 140, in add_bridge
      br.create()
    File "neutron/agent/common/ovs_lib.py", line 255, in create
      FAILMODE_SECURE))
    File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
      self.gen.next()
    File "/opt/stack/new/neutron/.tox/dsvm-functional/local/lib/python2.7/site-packages/ovsdbapp/api.py", line 94, in transaction
      self._nested_txn = None
    File "/opt/stack/new/neutron/.tox/dsvm-functional/local/lib/python2.7/site-packages/ovsdbapp/api.py", line 54, in __exit__
      self.result = self.commit()
    File "/opt/stack/new/neutron/.tox/dsvm-functional/local/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 57, in commit
      timeout=self.timeout)
  ovsdbapp.exceptions.TimeoutException: Commands [<ovsdbapp.schema.open_vswitch.commands.AddBridgeCommand object at 0x7fdcc274a9d0>, <ovsdbapp.schema.open_vswitch.commands.DbAddCommand object at 0x7fdcc274aad0>] exceeded timeout 10 seconds

  
  logstash query: http://logstash.openstack.org/#dashboard/file/logstash.json?query=build_name%3A%5C%22legacy-neutron-dsvm-functional%5C%22%20AND%20message%3A%5C%22ovsdbapp.exceptions.TimeoutException%3A%20Commands%5C%22%20AND%20tags%3Aconsole%20AND%20build_status%3AFAILURE

  
  In the logs we can see AddBridgeCommand result returning TRY_AGAIN over and over again:
  http://logs.openstack.org/70/531070/4/check/legacy-neutron-dsvm-functional/9aa7107/logs/dsvm-functional-logs/neutron.tests.functional.agent.test_dhcp_agent.DHCPAgentOVSTestCase.test_agent_mtu_set_on_interface_driver.txt.gz#_2018-01-08_04_20_06_754

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1741889/+subscriptions