dulwich-users team mailing list archive

[Augie Fackler <durin42@xxxxxxxxx>]

On Dec 26, 2010, at 2:27 PM, Jelmer Vernooij wrote:
> 
> On Sun, Dec 26, 2010 at 12:54:32PM -0500, Augie Fackler wrote:
>> On Dec 26, 2010, at 11:19 AM, Jelmer Vernooij wrote:
>>> On Sun, Dec 26, 2010 at 10:16:38AM -0500, Augie Fackler wrote:
>>>> On Dec 25, 2010, at 8:37 PM, Jelmer Vernooij wrote:
>>> The other test runners I can find - unittest, unittest2, testtools and trial all
>>> support running a specific test suite function, and can be run from a module
>>> ("python -m $TESTRUNNER dulwich.tests.test_suite"). This makes it easy to swap out one
>>> test runner for another, based on what is available and what the users personal
>>> preference is. As far as I can tell nose doesn't support this; I'd be glad to
>>> be proven wrong.
>> The *whole point* of nose is that you don't need test suites. It
>> intentionally ignores unittest.TestSuite entry points because it
>> assumes it'll find the tests via discovery.
> So there's no way to tweak what is tested without command-line magic?
> I can see some use in using discovery to find tests, but I'd like to
> be able to tweak what I do from within the Python code.

You can tweak things with setup.cfg, but you can't tell nose to test the test suites. Such a notion is completely counter to how nose works anyway (nose is intended to replace testsuites!)

>> Unfortunately, the
>> tutorial (which I didn't even know was a test suite until this set
>> of emails!) doesn't meet that description. That said, a tiny shim
>> function fixes the problem:
> The tutorial wasn't a testsuite until recently, when I found out that
> we introduced some API changes which broke it - so now we're testing it to make
> sure it will still work for users reading the docs.

Ah.

>> diff --git a/dulwich/tests/__init__.py b/dulwich/tests/__init__.py
>> index 2b3a652..259c043 100644
>> --- a/dulwich/tests/__init__.py
>> +++ b/dulwich/tests/__init__.py
>> @@ -126,6 +126,12 @@ def tutorial_test_suite():
>>         *tutorial_files)
> 
>> +def tutorial_tests():
>> +    for test in tutorial_test_suite():
>> +        print test # only here for debugging and proving that it works
>> +        test()
>> +
>> +
> Urgh.

I agree. For what it's worth, a Nose user would probably use the Trestle plugin to Nose (in case anyone following along is curious.)

>>>>> With the wider availability of python 2.7 - which includes a SkipTest
>>>>> class - I would also like to support using the standard Python
>>>>> test runner. If python 2.4, 2.5 or 2.6 is in use we can
>>>>> fallback to an alternative such as testtools.run, twisted.trial.run
>>>>> or unittest2.
>>>> IIRC, unittest2 comes with a discovery-based testrunner that's got most of the features of nose. Is that available in Python 2.7?
>>> See my patch - I'd like to use an explicit test collection function rather than
>>> discovery.
>> That's fine, can we use regular unittest for this instead of some
>> other module? I'm strongly resistant to
>> non-{unittest2,unittest,nose} options being necessary, even pre-2.7.
> I don't see why testtools is any more (or less) problematic in that
> regard than unittest2. For example, unittest2 has been packaged in
> Ubuntu since Maverick, while testtools has been available since
> Karmic.

It's increasingly likely that people will already have unittest2, because of situations like ours. Unless people are part of the Canonical universe, there's a near-zero chance they'll have testtools already. I'm not compelled one way or the other by the existence of ubuntu packages, since anyone that's doing dev work had best be comfortable with installing things from source anyway.

>>>> If you're really opposed to nose for some reason and are going to require a
>>>> 3rd party installable anyway, why not unittest2? It's the backport of 2.7's
>>>> testing system anyway, so nobody should particularly mind.
>>> For Python2.7 users there would no longer be a dependency on any 3rd party installables
>>> at all, only users of older versions of Python would have to install unittest2,
>>> testtools or trial (or perhaps nose if it can be used in the same way).
>> Personally, I'd like to depend on only the stdlib for testrunning on
>> 2.7, and then use unittest2 (as it's the official backport of 2.7's
>> unittest) for older systems.
> 
>> We can codify that in setup.py and make
>> it the One True Test System (and depending on unittest2 for old
>> Pythons means that we can continue using the normal SkipTest
>> approach *and* dispense with the madness in test/__init__ that tries
>> to get the "right" SkipTest. I'm happy to take on this work if
>> nobody objects.
> I can live with unittest2 as default fallback test runner but
> would like to keep the support for using testtools without the need to
> install unittest2. What do you think about the attached patch?

Should probably also list unittest2 in the test_requires stanza of the setup() call in setup.py if sys.version < (2,7).

Also: testtools can't handle normal unittest.TestCase subclasses? It feels like testtools should be adapting to unittest2 automatically, not users of testtools having to patch their code to deal with test skipping.

> Upstream Python appears to have gone with SkipTest as the name of the
> exception while we use TestSkipped. I don't have any particular
> preference for what we should use.

I guess it'd be nice to use the conventional name, but whatever. Not worth the churn.

>> Another option (which I don't like as much):
>> If you're willing to move to a decorator for test skipping instead
>> of raising an exception directly, we can have the decorator erase  
>> methods when test skipping isn't available and raise the appropriate
>> skip exception when it is, and then we can have the entire test
>> suite be runner-agnostic. Does that sound like a way that we can all
>> be reasonably happy?
> I don't like that one either, because it means the fact that tests
> were skipped (and why they were skipped) is hidden.

I agree, which was why I didn't like this option as much.

> 
> Cheers,
> 
> Jelmer
> <dulwich-unittest.run2.diff>