testtools-dev team mailing list archive

["Martin \[gz\]" <gzlist@xxxxxxxxxxxxxx>]

Martin [gz] has proposed merging lp:~gz/testtools/unicode_doctestmatches_764170 into lp:testtools with lp:~gz/testtools/wrap_utf8_terminals_804122 as a prerequisite.

Requested reviews:
  testtools developers (testtools-dev)

For more details, see:
https://code.launchpad.net/~gz/testtools/unicode_doctestmatches_764170/+merge/66674

Work towards trying to make DocTestMatches correct with non-ascii text. So far, bug 764170 is not actually fixed, because depending on the Python version there's all kinds of fun and interesting fallout.


Python 2.4, 2.5, and some 2.6.x versions
========================================

TestDocTestMatchesInterfaceUnicode.test_describe_difference fails because the AssertionError raised can't be stringified, as bug 804127 describes. The options here are mangling down to ascii, or using a AssertionError subclass that explicitly supports unicode.


Python 2.6 and 2.7
==================

TestDocTestMatchesInterfaceUnicode.test_describe_difference fails because doctest.OutputChecker.output_difference mangles any unicode down to str. It also does it in the most unhelpful way possible, using sys.stdout.encoding and from a module-global method innocently named '_indent' so it's very hard to revert the change to get the behaviour we want back again.

Blame: <http://hg.python.org/cpython/rev/d0c2b9c2babb>

Short of monkeypatching, I don't have any good ideas here.


Python 3
========

TestDocTestMatchesInterfaceUnicode.test_matches_match fails (unhelpfully) because on the statement `DocTestMatches("\xa7").match("\\xa7")` returns None, in other words, an escaped string matches its unescaped form. This basically makes doctest matchers useless for checking non-ascii text as it won't assert that you've got the escaping correct.

Blame: <http://hg.python.org/cpython/diff/da47e7e135ae/Lib/doctest.py>

This is fixable by overriding that method with a noop.

TestDocTestMatchesInterfaceUnicode.test_describe_difference fails because of the flipside, repr returns non-ascii strings by default. Adding another str_is_unicode conditional will fix this, the behaviour is somewhat reasonable.


Summary, there's a lot of needless pain in trying to reuse doctest.
-- 
https://code.launchpad.net/~gz/testtools/unicode_doctestmatches_764170/+merge/66674
Your team testtools developers is requested to review the proposed merge of lp:~gz/testtools/unicode_doctestmatches_764170 into lp:testtools.

=== modified file 'testtools/matchers.py'
--- testtools/matchers.py	2011-06-30 16:57:12 +0000
+++ testtools/matchers.py	2011-07-02 00:46:30 +0000
@@ -177,7 +177,7 @@
         return 'DocTestMatches(%r%s)' % (self.want, flagstr)
 
     def _with_nl(self, actual):
-        result = str(actual)
+        result = self.want.__class__(actual)
         if not result.endswith('\n'):
             result += '\n'
         return result

=== modified file 'testtools/tests/test_matchers.py'
--- testtools/tests/test_matchers.py	2011-06-30 11:42:40 +0000
+++ testtools/tests/test_matchers.py	2011-07-02 00:46:30 +0000
@@ -12,6 +12,8 @@
     )
 from testtools.compat import (
     StringIO,
+    str_is_unicode,
+    _u,
     )
 from testtools.matchers import (
     AfterPreproccessing,
@@ -112,6 +114,23 @@
         DocTestMatches("Ran 1 tests in ...s", doctest.ELLIPSIS))]
 
 
+class TestDocTestMatchesInterfaceUnicode(TestCase, TestMatchersInterface):
+
+    matches_matcher = DocTestMatches(_u("\xa7..."), doctest.ELLIPSIS)
+    matches_matches = [_u("\xa7"), _u("\xa7 more\n")]
+    matches_mismatches = ["\\xa7", _u("more \xa7"), _u("\n\xa7")]
+
+    str_examples = [("DocTestMatches(%s'\\xa7\\n')" % (
+            ["u", ""][str_is_unicode]),
+        DocTestMatches(_u("\xa7"))),
+        ]
+
+    describe_examples = [(
+        "Expected:\n    \\xa7\nGot:\n    a",
+        "a",
+        DocTestMatches(_u("\xa7"), doctest.ELLIPSIS))]
+
+
 class TestDocTestMatchesSpecific(TestCase):
 
     def test___init__simple(self):