launchpad-dev team mailing list archive

[Jonathan Lange <jml@xxxxxxxxxxxxx>]

On Wed, Oct 6, 2010 at 12:13 PM, Jonathan Lange <jml@xxxxxxxxxxxxx> wrote:
> On Wed, Oct 6, 2010 at 9:49 AM, Michael Nelson
> <michael.nelson@xxxxxxxxxxxxx> wrote:
>> I just caused a test-fix using ec2 land with a windmill test that
>> *did* fail, but the run was reported as successful (details below).
>>
...
>
> I'll try to figure something out with zope.testrunner now. I'll report
> back to the list when I've made progress or given up.

I'm sorry to say that the cause of this is a bug that I've been
meaning to fix for a long time.

I've just filed the bug here:
  https://bugs.edge.launchpad.net/zope.testing/+bug/655656

The guts of it is that zope.testrunner doesn't really have a way of
controlling the way layer errors are presented.

For our purposes, a patch like this will stop the stream corruption:
$ diff -u runner.py.real runner.py
--- runner.py.real	2010-10-06 12:36:36.000000000 +0100
+++ runner.py	2010-10-06 12:37:24.000000000 +0100
@@ -607,7 +607,7 @@
             output.tear_down_not_supported()
             if not optional:
                 raise CanNotTearDown(l)
-        else:
+        finally:
             output.stop_tear_down(time.time() - t)
         del setup_layers[l]

@@ -641,6 +641,7 @@
                         zope.testing.testrunner.debug.post_mortem(
                             sys.exc_info())
                 else:
+                    output.stop_set_up(time.time() - t)
                     raise

         output.stop_set_up(time.time() - t)


But that's a really crappy patch, and the right thing to do is fix
zope.testing as I describe in the bug report.

I've written a test that demonstrates the problem, which you can find
here: http://paste.ubuntu.com/507183/

Because of the zope.testing/zope.testrunner deprecation malarkey, and
the bluntness of its doctests and the fact that I've spent 2.5 hrs on
this already, I'm giving up here. I'll be more than happy to help
whoever takes up the baton.

jml