launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Tue, Nov 1, 2011 at 2:27 AM, Gary Poster <gary.poster@xxxxxxxxxxxxx> wrote:
> I have a couple of concerns about that approach (which in fact is what I tried initially with yuixhr).
>
> The first is that the main testprocess has no visibility of JS tests, or even of JS test cases.  It only has visibility into JS test suites as long as we follow the convention of one suite per file.

I don't quite understand that - the slave process is an appserver, not
a test runner; its the mechanized browser that is running js tests and
knows about everything, isn't it?

> somehow.  The approaches that I imagined for this seemed unnecessarily tricky and hand-wavy, though, and I don't have any new ideas in this regard.

We can probably work something out; or we can build up the code
muscles to let us do oops etc introspection in the js test code
itself. As yet I don't know whether having such duplication is
desirable (or not). We certainly have a rich testing environment in
Python and being unable to leverage that would be a bit sad.

That said, using the stock processlayercontroller doesn't imply the
parent test process knowning about individual js tests as it executes:
as long as either side is willing to do a reset-in-place its probably
all good. That won't be too hard to arrange IMO.

> The second concern is that I would like the interactive approach ("make run-testapp") and the non-interactive test-suite-integration approach to be as similar as possible so that it is easier to diagnose problems.  Indeed, one of the few differences between them (the test setup change caused by the INTERACTIVE_TESTS flag) added to the confusion in diagnosing this particular problem, and your fix was to eliminate that difference.

There is significant manual duplication between run-testapp and the
main test runner as it stands - we're going to have other cases of
duplication as we bring more microservices online - gpgverifyd,
mailbounced; jsoopsd etc.... I think there is a significant
maintenance burden in that duplication and we should put significant
resources into eliminating it.

>> For now, I'm landing a workaround: remove the 'INTERACTIVE_TESTS' flag
>> that was used to prevent rabbit starting - my code makes rabbit
>> desirable always, and we're going forward on that, not backwards.
>
> This sounds like a great direction for a solution.  I'll verify later that the parent process is not duplicating setup work.  (I plan to reinstate the flag itself in the Makefile so that I can use it for an additional interactive feature in one of my branches, but I won't let it affect the test setup.)

It certainly currently is duplicating effort when running a single
test, because the layer chosen to run the slave appserver is one
appropriate for running such appservers via the code path our other
tests which need an appserver do.

> So, I'm happy with where we are now with this, and what you've done.  We can discuss longer-term plans later, if you'd like to, but I think what you've done as a workaround is the right way forward.

Yes, I would like to discuss longer term plans, later :).

-Rob