launchpad-dev team mailing list archive

[Robert Collins <robert.collins@xxxxxxxxxxxxx>]

On Wed, Nov 3, 2010 at 6:44 AM, Deryck Hodge <deryck.hodge@xxxxxxxxxxxxx> wrote:
> On Tue, Nov 2, 2010 at 12:33 PM, Robert Collins
> <robertc@xxxxxxxxxxxxxxxxx> wrote:
>> I think this is a very interesting experiment that merits consideration.
>>
>> I'll answer in more detail at a later date, but I wanted to raise two
>> particular concerns now:
>> 1) should either this or the optional-reviews experiment let a
>> regression through it will be harder to clearly assess the cause.
>> Perhaps thats an unfounded worry, but I'd rather not run two
>> experiments both having roughly the same risk at the same time.
>>
>
> Sure, I can understand this concern.  I'm not sure it's hard to verify
> this independent of each other.  If we track regressions regardless of
> origin, then it seems simple to look and see if there's an unreviewed
> MP or if the change would have been covered by a page/windmill test.
> I don't think we should have the case of both because I didn't think
> we could land UI changes unreviewed.
>
> I do understand that it may be too much experiment at once, so if
> others feel that way, I'll certainly defer.  I thought it might be
> nice to run them together to see what kind of momentum we could build.
>  My personal hope is that others find the momentum addictive and want
> to maintain it. ;) :)  I don't want us to be reckless either but we
> seem to have an overabundance of caution on Launchpad.

We definitely need to build momentum and make better assessments of
risk. One big thing we can do is plan to deal-with-problems rather
than prevent-problems.

>> 2) Whats the rollback strategy?
>
> I see this as "re-enable the tests and go back to what we have now."
> I feel like that is too simple an answer for your question, so maybe
> I'm missing something that your concerned about?

Its just that tests tend to bitrot when they aren't run. It may be
hard to turn them back on.

-Rob