yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: keystone
       Status: Fix Committed => Fix Released

** Changed in: keystone
    Milestone: None => icehouse-3

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1251300

Title:
  test_cache_layer_*_crud depend on test execution order

Status in OpenStack Identity (Keystone):
  Fix Released

Bug description:
  The tests test_cache_layer_domain_crud, test_cache_layer_project_crud,
  and test_cache_layer_role_crud in
  keystone.tests.test_backend.IdentityTests may fail depending on which
  tests have run previously. In particular, if no tests have run
  previously, then they all fail. Here's an example running each test
  separately using the KVS backend:

  % tox -e py27  -- keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_role_crud
  py27 develop-inst-nodeps: /opt/stack/keystone
  py27 runtests: commands[0] | python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_role_crud
  running testr
  running=${PYTHON:-python} -m subunit.run discover \
  -t ./ ./keystone/tests \
  --list 
  running=${PYTHON:-python} -m subunit.run discover \
  -t ./ ./keystone/tests \
   --load-list /tmp/tmpCi8sRv
  ======================================================================
  FAIL: keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_role_crud
  tags: worker-0
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "keystone/tests/test_backend.py", line 2682, in test_cache_layer_role_crud
      self.assertDictEqual(role_ref, self.assignment_api.get_role(role_id))
    File "keystone/tests/core.py", line 423, in assertDictEqual
      self.assertEqual(d1, d2, msg)
    File "/opt/stack/keystone/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 322, in assertEqual
      self.assertThat(observed, matcher, message)
    File "/opt/stack/keystone/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 417, in assertThat
      raise MismatchError(matchee, matcher, mismatch, verbose)
  MismatchError: !=:
  reference = {'id': u'bf17c51445794329987eda59f9ffd64f',
   'name': u'ea760f7d48aa4c22a6dcaa54098fd2a5'}
  actual    = {'id': u'bf17c51445794329987eda59f9ffd64f',
   'name': u'815c02c26b3048308b1fb4be22412882'}
  ======================================================================
  FAIL: process-returncode
  tags: worker-0
  ----------------------------------------------------------------------
  Binary content:
    traceback (test/plain; charset="utf8")
  Ran 3 (+1) tests in 0.143s (+0.004s)
  FAILED (id=151, failures=2)
  error: testr failed (1)
  ERROR: InvocationError: '/opt/stack/keystone/.tox/py27/bin/python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_role_crud'
  ____________________________________________________________________________________ summary _____________________________________________________________________________________
  ERROR:   py27: commands failed
  % tox -e py27  -- keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_project_crud
  py27 develop-inst-nodeps: /opt/stack/keystone
  py27 runtests: commands[0] | python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_project_crud
  running testr
  running=${PYTHON:-python} -m subunit.run discover \
  -t ./ ./keystone/tests \
  --list 
  running=${PYTHON:-python} -m subunit.run discover \
  -t ./ ./keystone/tests \
   --load-list /tmp/tmp4QfzG9
  ======================================================================
  FAIL: keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_project_crud
  tags: worker-0
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "keystone/tests/test_backend.py", line 2636, in test_cache_layer_project_crud
      project, self.assignment_api.get_project(project_id))
    File "keystone/tests/core.py", line 480, in assertDictContainsSubset
      self.fail(self._formatMessage(msg, standardMsg))
    File "/usr/lib/python2.7/unittest/case.py", line 412, in fail
      raise self.failureException(msg)
  AssertionError: Mismatched values: 'name', expected: '4c6b94aaf6324a2c8d3da20fc19ced63', actual: u'aebeb924708f4cc7a326043d240be3af'
  ======================================================================
  FAIL: process-returncode
  tags: worker-0
  ----------------------------------------------------------------------
  Binary content:
    traceback (test/plain; charset="utf8")
  Ran 3 (+1) tests in 0.180s (+0.040s)
  FAILED (id=152, failures=2)
  error: testr failed (1)
  ERROR: InvocationError: '/opt/stack/keystone/.tox/py27/bin/python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_project_crud'
  ____________________________________________________________________________________ summary _____________________________________________________________________________________
  ERROR:   py27: commands failed
  % tox -e py27  -- keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_domain_crud 
  py27 develop-inst-nodeps: /opt/stack/keystone
  py27 runtests: commands[0] | python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_domain_crud
  running testr
  running=${PYTHON:-python} -m subunit.run discover \
  -t ./ ./keystone/tests \
  --list 
  running=${PYTHON:-python} -m subunit.run discover \
  -t ./ ./keystone/tests \
   --load-list /tmp/tmpY27vIC
  ======================================================================
  FAIL: keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_domain_crud
  tags: worker-0
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "keystone/tests/test_backend.py", line 2586, in test_cache_layer_domain_crud
      domain_ref, self.assignment_api.get_domain(domain_id))
    File "keystone/tests/core.py", line 480, in assertDictContainsSubset
      self.fail(self._formatMessage(msg, standardMsg))
    File "/usr/lib/python2.7/unittest/case.py", line 412, in fail
      raise self.failureException(msg)
  AssertionError: Mismatched values: 'name', expected: u'2414056b220a4e44ae11c9f4fb00376e', actual: u'a9e3a4ff47844a0f9f4911c9d7dbfa0e'
  ======================================================================
  FAIL: process-returncode
  tags: worker-0
  ----------------------------------------------------------------------
  Binary content:
    traceback (test/plain; charset="utf8")
  Ran 3 (+1) tests in 0.143s (-0.034s)
  FAILED (id=153, failures=2)
  error: testr failed (1)
  ERROR: InvocationError: '/opt/stack/keystone/.tox/py27/bin/python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_domain_crud'
  ____________________________________________________________________________________ summary _____________________________________________________________________________________
  ERROR:   py27: commands failed

  The results are similar for the SQL backend. I haven't tried the other
  backends.

  This test order dependency is particularly when running the tests in
  parallel since the test execution order is nondeterministic.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1251300/+subscriptions