testrepository-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

Something I've wanted to do for a while is be able to report on 'this
test failed with CLANG, this with GCC' or 'test X fails on python 2.6,
3.2'.

We have an isolated test API - the instance_provision/dispose_execute functions.

If we extend that by:
- adding a new variable $INSTANCE_PROFILE
- adding one or more ways to select profiles[1]
- namespacing test ids as they come in, and unwrapping the namespace
when we select them to run
- querying for test ids in all of the profiles[2]

Then we should be able to do some group-by stuff in the UI, or via
direct subunit commands, easily enough.

1]: I'm thinking a list_profiles function in .testr.conf, which could
be defined as "echo py26 py27 py32 py34" (for a constant set) or could
call out to a helper that inspects whats available, or whatever.
Probably with a command line to: -p py27,py32. I think perhaps that
that should do a set intersection check though, and error on missing
ones.

2]: I think the pain of querying N times will be outweighed by the
interesting things users may do with partitioned profiles.

-Rob

-- 
Robert Collins <rbtcollins@xxxxxx>
Distinguished Technologist
HP Converged Cloud