testtools-dev team mailing list archive

["Martin \[gz\]" <gzlist@xxxxxxxxxxxxxx>]

Review: Needs Fixing
On Python 3 (with lp:~gz/testtools/raises_regressions_675327 merged) I get three failures. All in TestMultiTestresultContract (nit, should that be TestMultiTest*R*esultContract?) the methods test_addError_is_failure, test_addFailure_is_failure, and test_addUnexpectedSuccess_was_successful all return True from `result.wasSuccessful()`.

There are actually four failures, but one is silent...


As noted by Martin Pool in bug 654474 comment 5 changing the outcome without also providing feedback is bad, so I think TextTestResult.stopTestRun also needs changing. Something along the lines of but prettier than:

=== modified file 'testtools/testresult/real.py'
--- testtools/testresult/real.py        2010-10-24 09:21:01 +0000
+++ testtools/testresult/real.py        2010-11-28 19:18:08 +0000
@@ -258,6 +261,8 @@
         stop = self._now()
         self._show_list('ERROR', self.errors)
         self._show_list('FAIL', self.failures)
+        for test in self.unexpectedSuccesses:
+            self.stream.write("%sUNEXPECTEDSUCCESS: %s\n%s" % (self.sep1, test.id(), self.sep2))
         self.stream.write("Ran %d test%s in %.3fs\n\n" %
             (self.testsRun, plural,
              self._delta_to_float(stop - self.__start)))
@@ -267,7 +272,7 @@
             self.stream.write("FAILED (")
             details = []
             details.append("failures=%d" % (
-                len(self.failures) + len(self.errors)))
+                len(self.failures) + len(self.errors) + len(self.unexpectedSuccesses)))
             self.stream.write(", ".join(details))
             self.stream.write(")\n")
         super(TextTestResult, self).stopTestRun()


The silent failure that should then be revealed on Python 3 also needs fixing:

=== modified file 'testtools/tests/test_compat.py'
--- testtools/tests/test_compat.py      2010-11-11 09:46:18 +0000
+++ testtools/tests/test_compat.py      2010-11-28 19:18:08 +0000
@@ -19,6 +19,7 @@
     )
 from testtools.matchers import (
     MatchesException,
+    Not,
     Raises,
     )

@@ -246,7 +247,7 @@
         if newio:
             self.expectFailure("Python 3 StringIO expects text not bytes",
                 self.assertThat, lambda: soutwrapper.write(self.uni),
-                Raises(MatchesException(TypeError)))
+                Not(Raises(MatchesException(TypeError))))
         soutwrapper.write(self.uni)
         self.assertEqual("pa???n", sout.getvalue())


Final nit:

+    def test_addUnexpectedSuccess_was_successful(self):
+        # addUnexpectedSuccess does not the test run in Python 2.7.

Verb?
-- 
https://code.launchpad.net/~jml/testtools/unexpected-success-2/+merge/42050
Your team testtools developers is subscribed to branch lp:testtools.