openstack team mailing list archive

[Duncan McGreggor <duncan@xxxxxxxxxxxxx>]

On Tue, Nov 29, 2011 at 12:21 PM, Soren Hansen <soren@xxxxxxxxxxx> wrote:
> It's been a bit over a week since I started this thread. So far we've
> agreed that running the test suite is too slow, mostly because there
> are too many things in there that aren't unit tests.
>
> We've also discussed my fake db implementation at length. I think
> we've generally agreed that it isn't completely insane, so that's
> moving along nicely.
>
> Duncan has taken the first steps needed to split the test suite into
> unit tests and everything else:
>
>   https://review.openstack.org/#change,1879
>
> Just one more core +1 needed. Will someone beat me to it? Only time
> will tell :) Thanks, Duncan!
>
> Anything else around unit testing anyone wants to get into The Great
> Big Plan[tm]?

Actually, yeah... one more thing :-)

Jay and I were chatting about organization of infrastructure last
night/this morning (on the review comments for the branch I
submitted). He said that I should raise a concern I expressed for
wider discussion: right now, tests are all piled into the tests
directory. Below are my thoughts on this.

I think such an approach is just fine for smaller projects; there's
not a lot there, and it's all pretty easy to find. For large projects,
this seems like not such a good idea for the following reasons:

 * tests are kept separate from the code they relate to
 * there are often odd test module file naming practices required
(e.g., nova/a/api.py and nova/b/api.py both needing test cases in
nova/tests/)
 * there's no standard exercised for whether a subpackage gets a
single test case module or whether it gets a test case subpackage
 * test modules tend to be very long (and thus hard to navigate) due
to the awkwardness of naming modules when all the code lives together
 * it makes it harder for newcomers to find code; when they live
together, it's a no-brainer

OpenStack is definitely not a small project, and as our test coverage
becomes more complete, these issues will have increased impact. I
would like to clean all of this up :-) And I'm volunteering to do the
work! Here's the sort of thing I envision, using nova.volume as an
example:

 * create nova/volume/tests
 * move all scheduler-related tests (there are several) from
nova/tests into nova/volume/tests
 * break out tests on a per-module basis (e.g., nova/volume/driver.py
would get the test module nova/volume/tests/test_driver.py, etc.)
 * for modules that have already been broken out at a more
fine-grained level, keep (smaller test case modules are nice!)
 * only nova/*.py files will have a test case module in nova/tests
 * bonus: update the test runner to print the full dotted path so it's
immediately (and visually) clear where one has to go to address any
failures

Given approval, this work would be done in its own blueprint. All this
work would be done in small chunks (probably one branch per module) so
that it will be easy to review and adjust the approach as needed.

Thoughts?

d

>
> --
> Soren Hansen        | http://linux2go.dk/
> Ubuntu Developer    | http://www.ubuntu.com/
> OpenStack Developer | http://www.openstack.org/
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp