testtools-dev team mailing list archive

["Martin \[gz\]" <gzlist@xxxxxxxxxxxxxx>]

Martin [gz] has proposed merging lp:~gz/testtools/wrap_utf8_terminals_804122 into lp:testtools.

Requested reviews:
  testtools developers (testtools-dev)
Related bugs:
  Bug #804122 in testtools: "UnicodeEncodeError in TextTestResult._show_list with UTF-8 terminal"
  https://bugs.launchpad.net/testtools/+bug/804122

For more details, see:
https://code.launchpad.net/~gz/testtools/wrap_utf8_terminals_804122/+merge/66635

Fixes a misunderstanding I had about how smart Python 2 was when writing unicode to streams with a unicode encoding. Only print does the right thing, the write method is still dumb and some parts of testtools use `stream.write` rather than `print >>stream`.

There's a slight risk here in that a utf-8 stream will now reject non-ascii bytestrings, but that's what my terminal has been doing since I introduced this code.
-- 
https://code.launchpad.net/~gz/testtools/wrap_utf8_terminals_804122/+merge/66635
Your team testtools developers is requested to review the proposed merge of lp:~gz/testtools/wrap_utf8_terminals_804122 into lp:testtools.

=== modified file 'testtools/compat.py'
--- testtools/compat.py	2011-04-24 21:48:56 +0000
+++ testtools/compat.py	2011-07-01 16:23:34 +0000
@@ -119,7 +119,9 @@
         return codecs.getwriter("ascii")(stream, "replace")
     if writer.__module__.rsplit(".", 1)[1].startswith("utf"):
         # The current stream has a unicode encoding so no error handler is needed
-        return stream
+        if sys.version_info > (3, 0):
+            return stream
+        return writer(stream)
     if sys.version_info > (3, 0):
         # Python 3 doesn't seem to make this easy, handle a common case
         try:

=== modified file 'testtools/tests/test_compat.py'
--- testtools/tests/test_compat.py	2010-11-29 00:21:59 +0000
+++ testtools/tests/test_compat.py	2011-07-01 16:23:34 +0000
@@ -15,6 +15,7 @@
     _detect_encoding,
     _get_source_encoding,
     _u,
+    str_is_unicode,
     unicode_output_stream,
     )
 from testtools.matchers import (
@@ -225,14 +226,18 @@
         unicode_output_stream(sout).write(self.uni)
         self.assertEqual([_b("pa?\xe8?n")], sout.writelog)
 
-    def test_unicode_encodings_not_wrapped(self):
-        """A unicode encoding is left unwrapped as needs no error handler"""
+    def test_unicode_encodings_wrapped(self):
+        """A unicode encoding is wrapped but needs no error handler"""
         sout = _FakeOutputStream()
         sout.encoding = "utf-8"
-        self.assertIs(unicode_output_stream(sout), sout)
-        sout = _FakeOutputStream()
-        sout.encoding = "utf-16-be"
-        self.assertIs(unicode_output_stream(sout), sout)
+        uout = unicode_output_stream(sout)
+        self.assertEqual(uout.errors, "strict")
+        uout.write(self.uni)
+        if str_is_unicode:
+            expected = self.uni
+        else:
+            expected = _b("pa\xc9\xaa\xce\xb8\xc9\x99n")
+        self.assertEqual([expected], sout.writelog)
 
     def test_stringio(self):
         """A StringIO object should maybe get an ascii native str type"""