launchpad-dev team mailing list archive

[Deryck Hodge <deryck.hodge@xxxxxxxxxxxxx>]

Hi, Ian.

On Tue, Nov 2, 2010 at 6:19 PM, Ian Booth <ian.m.booth@xxxxxxxxx> wrote:
> Tim beat me to the keyboard here, so let me add my vote to what he says.
> Without Windmill, how do we propose to test stuff like AJAX
> functionality which AIUI we must test in an actual browser? I think we
> expect the amount of AJAX usage in Launchpad to increase? So taking away
> a means to test that would be a problem I would think.
>
> I also agree with other comments that it is less than desirable to
> remove tests which do serve a useful purpose. Windmill tests are indeed
> integration tests which exercise the entire stack end-end. The value of
> this should not be underestimated for two reasons: 1. we do not seem to
> have 100% coverage using alternate testing mechanisms and this type of
> test is a great safety net since a breakage that may not be caught
> elsewhere will often show up here; 2. full end-end system testing is
> often the only way to catch certain classes of error which are difficult
> to pick up with just unit or module tests. Add to that that as stated
> above, there's currently no other alternative for AJAX testing etc.
>

I don't think the Windmill tests are integration or end to end tests.
That's what they aspire to be. :)  They are at best interaction tests.
 Also, I share you perspective in terms of the ideal for testing, but
Windmill does not provide this ideal.  I do agree we should test AJAX
and JS interactions, though, which is why I'm in favor of leaving
Windmill to run yui unit tests.  If we came up with a clear story for
how to do reliable, useful, and fast end to end testing, I would of
course support that, but I don't think Windmill can ever be that.
That's a different issue, IMHO, then asking if the tests as they are
now have any value.

Cheers,
deryck

-- 
Deryck Hodge
https://launchpad.net/~deryck
http://www.devurandom.org/