openstack team mailing list archive

[Monty Taylor <mordred@xxxxxxxxxxxx>]

On 12/30/2011 02:09 PM, Jay Pipes wrote:
> On Fri, Dec 30, 2011 at 5:01 PM, Monty Taylor <mordred@xxxxxxxxxxxx> wrote:
>> Hey all!
>>
>> I've been poking around at a few things over the last week or so, and I
>> thought I'd put some thoughts out there for everyone.
>>
>> There are two issues that have been driving the work:
>>
>> 1) We need to be able to sensibly run unit tests against multiple python
>> versions (specifically 2.6 and 2.7 at the moment)
>>
>> 2) I'd like to reduce the amount of boilerplate or (even worse) modified
>> boiler plate code that we have copied around in all of our repos. (it
>> makes incorporating new projects painful)
>>
>> I could solve (1) with matrix builds in jenkins, but that doesn't help
>> put any tooling in the hands of devs so that they can run those tests
>> locally before submitting.
>>
>> To this end, (and thanks to a config file that was in
>> python-keystoneclient) I've been exploring the tool tox. tox actually
>> addresses both of the above points out of the box. tox allows you to
>> specify multiple virtualenv environments you want to run against, and
>> will happily do the testing in each of them. It's testing tool agnostic
>> (essentially, it does pretty much the same thing that install_venv.py
>> and with_venv.py do, except across multiple environments)
> 
> w00t. Looking forward to simplifying and unifying the various
> run_tests and venv stuff in the core projects...
> 
>> An example tox.ini file for glance looks like this:
>>
>> [tox]
>> envlist = py26,py27
>>
>> [testenv]
>> deps = -r{toxinidir}/tools/pip-requires
>> commands = /bin/bash run_tests.sh -N
> 
> I don't think that the -N makes any sense above, since that indicates
> to run WITHOUT a virtualenv. ;) See my similar comment on the Glance
> tox merge proposal...

That will eventually go away ... it's just because we're using tox in
this case to manage the virtualenv, so we don't want run_tests.sh to
create one.

>> [testenv:pep8]
>> commands = pep8 --repeat glance
>>
>> If you run:
>>
>> # tox
>>
>> You get run_tests.sh -N run against a python2.6 and a python2.7 virtual
>> env. If you run:
>>
>> # tox -e py27
>>
>> You get just tests against 2.7. If you run:
>>
>> # tox -e pep8
>>
>> You get the pep8 run in a venv.
>>
>> Because we can reference our existing pip-requires files, we can add the
>> tox.ini files without really impacting anything else for the purposes of
>> kicking the tires.
>>
>> What I'd like to do going forward is this;
>>
>> - propose some patches to projects which add a tox.ini file and send out
>> some instructions for people to try using it
>>
>> - if we like it, modify the jenkins jobs to start invoking tox instead
>> of run_tests,sh
>>
>> - eventually, if we're happy, make run_tests.sh just a simple wrapper
>> script which calls tox with a few parameters and remove install_venv.sh
>> and with_venv.sh
>>
>> Upstream for tox has been pretty responsive so far - and I'm working on
>> a patch that we would need to land in order for us to be able to do our
>> venv pre-caching in jenkins properly - so any features we might need in
>> it should be easy enough to deal with. However, so far I haven't found
>> anything we currently do that it can't pretty much already handle.
>>
>> Any thoughts or feedback? (Most importantly, are there any drastic
>> objections to further investigation of it as a tool to use here?)
> 
> No objections from me.

w00t