launchpad-dev team mailing list archive

[Jonathan Lange <jml@xxxxxxxxxxxxx>]

On Wed, Nov 3, 2010 at 2:01 AM, Deryck Hodge <deryck.hodge@xxxxxxxxxxxxx> wrote:
> Hi, Tim.
>
> Thanks for weighing in.
>
> On Tue, Nov 2, 2010 at 5:50 PM, Tim Penhey <tim.penhey@xxxxxxxxxxxxx> wrote:
>> On Wed, 03 Nov 2010 07:18:32 Gary Poster wrote:
>> The page tests often test workflows, which are getting more out of date with
>> the addition of more ajax stuff, but I think it still has some, but limited,
>> use.
>
> But this is my whole point exactly....  your own words: you *think*
> they have *some* use.
>

And a very good point it is too. I also challenge the usefulness of these tests.

> We don't know the value of these tests or how much usefulness they
> provide.  We don't know the amount of coverage we actually get from
> them.  We have half or better of a four+ hour test run for these tests
> and we have no way to assess the value of this cost.  We have
> developers avoiding page tests and windmill at all costs and no clear
> story of even why we use them or what we expect from them.  It seems
> to me a short and controlled experiment is an easy way to get some
> data on these questions.
>

Would you not get the same data by running the page & windmill tests
separately as Gary suggests?

jml