testtools-dev team mailing list archive

["Martin \[gz\]" <gzlist@xxxxxxxxxxxxxx>]

Public bug reported:

Currently, assertThat seems like a usability regression. To demonstrate,
compare the same test from wikkid failing twice, one on a testtools
subclass where assertThat is used, and one also subclassing the Bazaar
case class which uses their assertEquals method rather than the
testtools implementation:


Tests running...
======================================================================
FAIL: wikkid.tests.test_volatile_filestore.TestVolatileFileStore.test_mimetype
----------------------------------------------------------------------
Text attachment: traceback
------------
Traceback (most recent call last):
  File "C:\bzr\testtools\testtools\runtest.py", line 143, in _run_user
    return fn(*args)
  File "C:\bzr\testtools\testtools\testcase.py", line 539, in _run_test_method
    return self._get_test_method()()
  File "C:\bzr\wikkid\wikkid\tests\filestore.py", line 68, in test_mimetype
    self.assertEqual(
  File "C:\bzr\testtools\testtools\testcase.py", line 309, in assertEqual
    self.assertThat(observed, matcher)
  File "C:\bzr\testtools\testtools\testcase.py", line 384, in assertThat
    self.fail('Match failed. Matchee: "%s"\nMatcher: %s\nDifference: %s\n'
AssertionError: Match failed. Matchee: "None"
Matcher: Equals('text/x-c++src')
Difference: 'text/x-c++src' != None

------------
======================================================================
FAIL: wikkid.tests.test_bzr_filestore.TestBzrFileStore.test_mimetype
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\bzr\wikkid\wikkid\tests\filestore.py", line 68, in test_mimetype
    self.assertEqual(
AssertionError: not equal:
a = 'text/x-c++src'
b = None

Ran 114 tests in 5.078s

FAILED (failures=2)


Robert noted two main differences:
# The extra levels of traceback, which he finds useful as he works on the test code itself. I'd suggest most people don't, and only care about their tests. The <lp:~gz/testtools/cleaner_tracebacks> branch would fix this, but makes life harder for the Roberts of the world and there doesn't seem much interest in landing it.
# The formatting is different. Robert felt there isn't much different between them, but the assertThat version prints the expected and actual output twice and uses longer more complicated words. It should make it clearer which is the expected value however, if the test is correctly written. Also, exarkun noted he liked being able to paste the test output into a repl which the Bazaar version enables.

See also bug 660852 as a wider output verbosity issue.

** Affects: testtools
     Importance: Undecided
         Status: New

-- 
assertThat style gives overly verbose output
https://bugs.launchpad.net/bugs/675323
You received this bug notification because you are a member of testtools
developers, which is subscribed to testtools.

Status in testtools: New

Bug description:
Currently, assertThat seems like a usability regression. To demonstrate, compare the same test from wikkid failing twice, one on a testtools subclass where assertThat is used, and one also subclassing the Bazaar case class which uses their assertEquals method rather than the testtools implementation:


Tests running...
======================================================================
FAIL: wikkid.tests.test_volatile_filestore.TestVolatileFileStore.test_mimetype
----------------------------------------------------------------------
Text attachment: traceback
------------
Traceback (most recent call last):
  File "C:\bzr\testtools\testtools\runtest.py", line 143, in _run_user
    return fn(*args)
  File "C:\bzr\testtools\testtools\testcase.py", line 539, in _run_test_method
    return self._get_test_method()()
  File "C:\bzr\wikkid\wikkid\tests\filestore.py", line 68, in test_mimetype
    self.assertEqual(
  File "C:\bzr\testtools\testtools\testcase.py", line 309, in assertEqual
    self.assertThat(observed, matcher)
  File "C:\bzr\testtools\testtools\testcase.py", line 384, in assertThat
    self.fail('Match failed. Matchee: "%s"\nMatcher: %s\nDifference: %s\n'
AssertionError: Match failed. Matchee: "None"
Matcher: Equals('text/x-c++src')
Difference: 'text/x-c++src' != None

------------
======================================================================
FAIL: wikkid.tests.test_bzr_filestore.TestBzrFileStore.test_mimetype
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\bzr\wikkid\wikkid\tests\filestore.py", line 68, in test_mimetype
    self.assertEqual(
AssertionError: not equal:
a = 'text/x-c++src'
b = None

Ran 114 tests in 5.078s

FAILED (failures=2)


Robert noted two main differences:
# The extra levels of traceback, which he finds useful as he works on the test code itself. I'd suggest most people don't, and only care about their tests. The <lp:~gz/testtools/cleaner_tracebacks> branch would fix this, but makes life harder for the Roberts of the world and there doesn't seem much interest in landing it.
# The formatting is different. Robert felt there isn't much different between them, but the assertThat version prints the expected and actual output twice and uses longer more complicated words. It should make it clearer which is the expected value however, if the test is correctly written. Also, exarkun noted he liked being able to paste the test output into a repl which the Bazaar version enables.

See also bug 660852 as a wider output verbosity issue.