openstack team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

On Fri, Dec 30, 2011 at 5:01 PM, Monty Taylor <mordred@xxxxxxxxxxxx> wrote:
> Hey all!
>
> I've been poking around at a few things over the last week or so, and I
> thought I'd put some thoughts out there for everyone.
>
> There are two issues that have been driving the work:
>
> 1) We need to be able to sensibly run unit tests against multiple python
> versions (specifically 2.6 and 2.7 at the moment)
>
> 2) I'd like to reduce the amount of boilerplate or (even worse) modified
> boiler plate code that we have copied around in all of our repos. (it
> makes incorporating new projects painful)
>
> I could solve (1) with matrix builds in jenkins, but that doesn't help
> put any tooling in the hands of devs so that they can run those tests
> locally before submitting.
>
> To this end, (and thanks to a config file that was in
> python-keystoneclient) I've been exploring the tool tox. tox actually
> addresses both of the above points out of the box. tox allows you to
> specify multiple virtualenv environments you want to run against, and
> will happily do the testing in each of them. It's testing tool agnostic
> (essentially, it does pretty much the same thing that install_venv.py
> and with_venv.py do, except across multiple environments)

w00t. Looking forward to simplifying and unifying the various
run_tests and venv stuff in the core projects...

> An example tox.ini file for glance looks like this:
>
> [tox]
> envlist = py26,py27
>
> [testenv]
> deps = -r{toxinidir}/tools/pip-requires
> commands = /bin/bash run_tests.sh -N

I don't think that the -N makes any sense above, since that indicates
to run WITHOUT a virtualenv. ;) See my similar comment on the Glance
tox merge proposal...

> [testenv:pep8]
> commands = pep8 --repeat glance
>
> If you run:
>
> # tox
>
> You get run_tests.sh -N run against a python2.6 and a python2.7 virtual
> env. If you run:
>
> # tox -e py27
>
> You get just tests against 2.7. If you run:
>
> # tox -e pep8
>
> You get the pep8 run in a venv.
>
> Because we can reference our existing pip-requires files, we can add the
> tox.ini files without really impacting anything else for the purposes of
> kicking the tires.
>
> What I'd like to do going forward is this;
>
> - propose some patches to projects which add a tox.ini file and send out
> some instructions for people to try using it
>
> - if we like it, modify the jenkins jobs to start invoking tox instead
> of run_tests,sh
>
> - eventually, if we're happy, make run_tests.sh just a simple wrapper
> script which calls tox with a few parameters and remove install_venv.sh
> and with_venv.sh
>
> Upstream for tox has been pretty responsive so far - and I'm working on
> a patch that we would need to land in order for us to be able to do our
> venv pre-caching in jenkins properly - so any features we might need in
> it should be easy enough to deal with. However, so far I haven't found
> anything we currently do that it can't pretty much already handle.
>
> Any thoughts or feedback? (Most importantly, are there any drastic
> objections to further investigation of it as a tool to use here?)

No objections from me.

Cheers,
-jay