launchpad-dev team mailing list archive

[Robert Collins <robert.collins@xxxxxxxxxxxxx>]

So from what I can tell, *all* the windmill failures have been one of:
 - incorrect tests
 - failed landings - landed without ec2land and that then *never worked*

I'm not aware of /any/ intermittent /correct/ tests.

We could easily have the same issues with intermitted results were we
to use threaded helpers and not synchronise in our code.

When a test fails in trunk we need to - and I believe this is policy already:
 - file a bug
 - diagnose the cause
 - take remedial action - in order of preference:
  - fix the test
  - revert the landing that broke it/added the broken test
  - disable the test

This is obviously related to the discussion about how much value we
get from the tests, but I've not seen any data saying that *windmill*
is our problem. It seems to be hard to use correctly, and we seem to
be failing to use it correctly in at least some places. Which sucks
:(.

-Rob