launchpad-dev team mailing list archive

["Francis J. Lacoste" <francis.lacoste@xxxxxxxxxxxxx>]

On November 9, 2010, Julian Edwards wrote:
> On Tuesday 09 November 2010 11:47:23 Graham Binns wrote:
> > On 9 November 2010 11:13, Julian Edwards <julian.edwards@xxxxxxxxxxxxx>
> > 
> > > I think we should revert back to a separate buildbot run for Windmill
> > > tests. Do we really need to run them for every single landed branch?
> > 
> > +1. I don't think that Windmill tests are helping out for anything
> > other than changes which affect user interaction. As has been
> > discussed in another thread, maybe we should consider them a barrier
> > to deployment rather than landing.
> 
> Yep, I just caught up with that thread after sending this email.  (Yay for
> email armageddon whem coming back from holidays)
> 
> I'd like to refine the suggestion to say that I'd like to see Windmill AND
> pagetests running in an advisory mode only on a separate buildbot, and not
> as a deployment blocker.
> 
> This would enable us to run Deryck's experiment while at the same time
> continually assessing the damage from bitrot (and fixing quickly if we
> think failures are genuine).
> 

We used to run the windmill tests in a separate buildbot and they did bitrot 
and we only got to fixing them when it became a blocker for landing. Why would 
it be different this time?

That's why it should be a blocker for deployment. Either these tests have 
value and are preventing regressions in production, or they don't and we 
should just kill them.

The whole point of Deryck's experiment is to assess this cost/benefit, and 
while doing the experiment, its important to ensure reversibility which means 
no bitrot.

-- 
Francis J. Lacoste
francis.lacoste@xxxxxxxxxxxxx

Attachment: signature.asc
Description: This is a digitally signed message part.