testtools-dev team mailing list archive

[Jonathan Lange <jml@xxxxxxxxx>]

I need to recap my understanding of the situation in order to be able to fix it. Please correct me if I've missed anything, got anything wrong or added unnecessary stuff:
  * We've fixed the problem of addUnexpectedSuccess falling back to addSuccess. It now falls back to addFailure.
  * An unexpected success in testtools will not fail the test run
  * Unexpected successes will not be printed by TextTestRunner

Looking at unittest in Python 2.7:
  * An unexpected success will not fail the test run
  * Unexpected successes will be printed in the incremental output of TextTestRunner
  * Unexpected successes will *not* be included in the summary of TextTestRunner

I'm genuinely not sure whether we should differ from unittest here. Rob,
what are your thoughts? I'll try to knock up a patch in the mean time.

-- 
An unexpectedSuccess is like a failure not a success
https://bugs.launchpad.net/bugs/654474
You received this bug notification because you are a member of testtools
developers, which is subscribed to testtools.

Status in Bazaar Version Control System: Confirmed
Status in SubUnit: New
Status in testtools: In Progress

Bug description:
When the function passed to expectedFailure did not raise an assertion, bzr used to do:

    self.fail('Unexpected success.  Should have failed: %s' % reason)

With testtools, bazaar now treats that as a success, as jam noted in February:
<https://lists.ubuntu.com/archives/bazaar/2010q1/066898.html>

And in September he pointed out the same thing, but with subunit:
<https://lists.ubuntu.com/archives/bazaar/2010q3/070192.html>