testrepository-dev team mailing list archive

[Jelmer Vernooij <jelmer@xxxxxxxxx>]

On Fri, Jul 31, 2015 at 01:14:57PM +1200, Robert Collins wrote:
> On 31 July 2015 at 13:07, Jelmer Vernooij <jelmer@xxxxxxxxx> wrote:
> >
> >
> > On 30/07/15 11:59, Robert Collins wrote:
> 
> >> So it seems like a subset of profiles. How is it different?
> >
> > The test runner takes care of the environments, not the test tool (like
> > e.g. testr). The profile is also encoded in the test name when reporting
> > results.
> 
> So testr is both just an analysis tool (feed stuff into load, query it
> later) and a runner-runner, where it requests things to be run.
> 
> The impact of profiles on load will be how we identify the profile a
> test is coming from.
> 
> The impact of profiles on the meta-runner is scheduling: asking for
> tests in all applicable profiles.
> 
> The impact on reporting is that we'll want to show it - and I think we
> should aggregate by default.
Aggregate in what direction? Per environment or per test?
> 
> So it sounds like within the context of load+query profiles==environments.
Yeah, that sounds about right.

> > Each environment has some setup and teardown code associated with it.
> > In particular, our test runner knows how to reuse existing servers when
> > setting up new environments.
> 
> So a single server process can serve several different environments?
> What describes the difference between environments?
You can e.g. run the tests against a standalone server, against a domain
controller in a active directory domain, against a workstation in
an active directory domain, against a domain controller in a foreign domain
that trusts yours, etc.

If you run a test against a DC, we only bring up a DC. If you then also
run a test against a workstation we'll bring up a workstation server
and reuse the existing DC.

If you just run a single test against a DC, we only bring that up.

This matters, because starting the servers in particular is slow, and we'd like
to able to rapidly run individual tests (either against a single
environment or multiple).

Cheers,

Jelmer