yahoo-eng-team team mailing list archive

[Brant Knudson <bknudson@xxxxxxxxxx>]

Public bug reported:


We've got a lot of tests that depend on how long the test takes to run. Tests can take a long time just because you have a slow or overloaded system, or maybe you're trying to step through it with the debugger.

The tests that fail generally don't care about the time and aren't
attempting to verify performance, but still require that the test run
quickly enough.

Tests shouldn't depend on the wall clock time, just like they shouldn't
depend on any external factors.

Here's an example of a failing test:

keystone.tests.unit.test_auth.AuthWithRemoteUser.test_scoped_remote_authn
-------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "keystone/tests/unit/test_auth.py", line 741, in test_scoped_remote_authn
        enforce_audit_ids=False)
      File "keystone/tests/unit/test_auth.py", line 104, in assertEqualTokens
        timeutils.parse_isotime(b['access']['token']['expires']))
      File "keystone/tests/unit/core.py", line 521, in assertCloseEnoughForGovernmentWork
        self.assertTrue(abs(a - b).seconds <= delta, msg)
      File "/home/jenkins/workspace/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/case.py", line 678, in assertTrue
        raise self.failureException(msg)
    AssertionError: False is not true : 2015-06-11 13:34:46+00:00 != 2015-06-11 13:34:50+00:00 within 3 delta

It took 4 seconds rather than 3.

** Affects: keystone
     Importance: Undecided
     Assignee: Brant Knudson (blk-u)
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1464366

Title:
  unit tests fail based on wall clock time

Status in OpenStack Identity (Keystone):
  New

Bug description:
  
  We've got a lot of tests that depend on how long the test takes to run. Tests can take a long time just because you have a slow or overloaded system, or maybe you're trying to step through it with the debugger.

  The tests that fail generally don't care about the time and aren't
  attempting to verify performance, but still require that the test run
  quickly enough.

  Tests shouldn't depend on the wall clock time, just like they
  shouldn't depend on any external factors.

  Here's an example of a failing test:

  keystone.tests.unit.test_auth.AuthWithRemoteUser.test_scoped_remote_authn
  -------------------------------------------------------------------------

  Captured traceback:
  ~~~~~~~~~~~~~~~~~~~
      Traceback (most recent call last):
        File "keystone/tests/unit/test_auth.py", line 741, in test_scoped_remote_authn
          enforce_audit_ids=False)
        File "keystone/tests/unit/test_auth.py", line 104, in assertEqualTokens
          timeutils.parse_isotime(b['access']['token']['expires']))
        File "keystone/tests/unit/core.py", line 521, in assertCloseEnoughForGovernmentWork
          self.assertTrue(abs(a - b).seconds <= delta, msg)
        File "/home/jenkins/workspace/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/case.py", line 678, in assertTrue
          raise self.failureException(msg)
      AssertionError: False is not true : 2015-06-11 13:34:46+00:00 != 2015-06-11 13:34:50+00:00 within 3 delta

  It took 4 seconds rather than 3.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1464366/+subscriptions