dulwich-users team mailing list archive

[Augie Fackler <durin42@xxxxxxxxx>]

On Dec 26, 2010, at 11:19 AM, Jelmer Vernooij wrote:
> 
> On Sun, Dec 26, 2010 at 10:16:38AM -0500, Augie Fackler wrote:
>> On Dec 25, 2010, at 8:37 PM, Jelmer Vernooij wrote:
>>> There is an option to enable the running of doctests, but with that the
>>> setup/teardown function for the doctests as we specify them in
>>> our tutorial_test_suite() function don't get used. Is anybody aware of
>>> a way to get nose to use a test-suite function? I briefly looked over
>>> the documentation but couldn't find anything.
>> Not that I can think of offhand. I can ask the author of Nose if you want
>> (he's a friend), but I think the usual way of doing this is to explicitly
>> call setup/teardown in your doctest. Note that I think that's the usual way
>> even with unittest and friends. Alternatively, we can probably hook up enough
>> of the doctest plumbing in a real test method, but I think I'd rather be able
>> to write small doctests when it makes sense.
> As this discussion shows everybody has a favorite test runner. I'd like to support using
> either the default python test runner (so we can drop an external dependency) and
> allow users to specify a custom test runner easily.
> 
> The other test runners I can find - unittest, unittest2, testtools and trial all
> support running a specific test suite function, and can be run from a module
> ("python -m $TESTRUNNER dulwich.tests.test_suite"). This makes it easy to swap out one
> test runner for another, based on what is available and what the users personal
> preference is. As far as I can tell nose doesn't support this; I'd be glad to
> be proven wrong.

The *whole point* of nose is that you don't need test suites. It intentionally ignores unittest.TestSuite entry points because it assumes it'll find the tests via discovery. Unfortunately, the tutorial (which I didn't even know was a test suite until this set of emails!) doesn't meet that description. That said, a tiny shim function fixes the problem:

diff --git a/dulwich/tests/__init__.py b/dulwich/tests/__init__.py
index 2b3a652..259c043 100644
--- a/dulwich/tests/__init__.py
+++ b/dulwich/tests/__init__.py
@@ -126,6 +126,12 @@ def tutorial_test_suite():
         *tutorial_files)
 
 
+def tutorial_tests():
+    for test in tutorial_test_suite():
+        print test # only here for debugging and proving that it works
+        test()
+
+
 def nocompat_test_suite():
     result = unittest.TestSuite()
     result.addTests(self_test_suite())

(nose then detects the bare test function and runs it, see for yourself with 'nosetests -vs')

> 
>>> With the wider availability of python 2.7 - which includes a SkipTest
>>> class - I would also like to support using the standard Python
>>> test runner. If python 2.4, 2.5 or 2.6 is in use we can
>>> fallback to an alternative such as testtools.run, twisted.trial.run
>>> or unittest2.
>> IIRC, unittest2 comes with a discovery-based testrunner that's got most of the features of nose. Is that available in Python 2.7?
> See my patch - I'd like to use an explicit test collection function rather than
> discovery.

That's fine, can we use regular unittest for this instead of some other module? I'm strongly resistant to non-{unittest2,unittest,nose} options being necessary, even pre-2.7.

> 
>>> The attached patch changes the Makefile to use unittest's test runner if it is
>>> recent enough and otherwise fall back to testtools.
>>> 
>>> Is anybody particularly attached to nose for some reason? Other thoughts?
>> Well, I use nose on a *ton* of projects, many of them not mine, and I've
>> literally never heard of testtools until this email. That alone seems like a
>> nice reason to prefer nose over testtools (I'd be resistant to testtools,
>> given that it doesn't seem to have a ton of mindshare) Nose also comes with
>> the SkipTest functionality we need on older Pythons.
> For me it's the other way around. Bazaar, Launchpad and Samba all use testtools. Other
> than Dulwich I don't use nose anywhere.
> 
>> If you're really opposed to nose for some reason and are going to require a
>> 3rd party installable anyway, why not unittest2? It's the backport of 2.7's
>> testing system anyway, so nobody should particularly mind.
> For Python2.7 users there would no longer be a dependency on any 3rd party installables
> at all, only users of older versions of Python would have to install unittest2,
> testtools or trial (or perhaps nose if it can be used in the same way).

Personally, I'd like to depend on only the stdlib for testrunning on 2.7, and then use unittest2 (as it's the official backport of 2.7's unittest) for older systems. We can codify that in setup.py and make it the One True Test System (and depending on unittest2 for old Pythons means that we can continue using the normal SkipTest approach *and* dispense with the madness in test/__init__ that tries to get the "right" SkipTest. I'm happy to take on this work if nobody objects.

Another option (which I don't like as much):
If you're willing to move to a decorator for test skipping instead of raising an exception directly, we can have the decorator erase methods when test skipping isn't available and raise the appropriate skip exception when it is, and then we can have the entire test suite be runner-agnostic. Does that sound like a way that we can all be reasonably happy?

> 
>> That, or we can put together something that's not discovery-based at all.
>> We've got that set up for hgsubversion, but I still use nose anyway because
>> it adds so many nice features on top of unittest.
> Yeah, that's what my patch does - it calls dulwich.tests.test_suite. How do you use
> something like that together with nose in hgsubversion?

The tests merely work with nose in hgsubversion. The test suites only exist for the benefit of unittest.