launchpad-dev team mailing list archive

[William Grant <william.grant@xxxxxxxxxxxxx>]

On 02/03/11 20:06, Deryck Hodge wrote:
> On Wed, Mar 2, 2011 at 1:00 AM, Jonathan Lange <jml@xxxxxxxxxxxxx> wrote:
>> On Wed, Mar 2, 2011 at 3:15 AM, William Grant
>> <william.grant@xxxxxxxxxxxxx> wrote:
>>> Hi all,
>>>
>>> After several weeks of hangs and spurious failures with no resolution,
>>> I've removed WindmillLayer from the default test suite. This means that
>>> buildbot and ec2 should be reliable and fastish again.
>>>
>>> However, all is not lost. WindmillLayer now has its own Jenkins job at
>>> https://hudson.wedontsleep.org/job/windmill. I'm not sure if it will
>>> fail as often... I guess we'll see.
>>
>> Thanks. I think that this was the right thing to do.
>>
> 
> Like Jono, I think this was the right thing to do.  Thanks, William.
> I really do appreciate your effort to understand Windmill and
> frustrations at not being able to do anything about it and the failing
> test runs.
> 
> I do think, though, that we need a better long term solution than
> putting the tests off on Jenkins.  We know from past experience that
> Windmill tests run outside of our normal test run will bit rot.  We
> need to develop more experience with debugging Windmill tests or
> making Windmill itself stable, or we need to aggressively move off the
> tool.

Certainly. This is only a short-term solution. And one that I am not at
all happy with.

I am hoping that this will let us see the tests that are flaky -- and
maybe even help us debug the hangs -- without blocking the pipeline for
several hours each time it occurs. Once we have Windmill itself and a
reasonable set of tests stable, we should turn it back on everywhere.

William.

Attachment: signature.asc
Description: OpenPGP digital signature