launchpad-dev team mailing list archive

["Francis J. Lacoste" <francis.lacoste@xxxxxxxxxxxxx>]

On October 19, 2010, Maris Fogels wrote:
> On 10/18/2010 09:53 PM, Robert Collins wrote:
> > Process Overview
> > Some reviews are not needed and eligible developers can choose to have
> > branches land without formal review.
> > 
> > Process Description
> 
> ...
> 
> > Production problems which are tracked to unreviewed landings will also
> > be input into assessing the experiment.
> 
> Since it looks like we will be going ahead with the experiment, do we need
> to flesh out this point about investigating production problems?   Besides
> the random review audits, it represents the only empirical evidence by
> which we can judge that the experiment is causing problems.
> 
> It sounds like this point requires a new small process to look at
> production problems (which ones?), trace their origin, say if it was an
> unreviewed change that was the cause.  This requires discipline.

It's true that we don't have this discipline in place. Formally, we only do 
root-cause-analysis in incident report, and even there we often don't.

If we were tracking escaped defects, we could just compared the numbers before 
and after the experiment. But were aren't. There is no way currently to know 
the number of reported bugs we have. The bug tracker contains tasks, tech-
debt, future-todos, wishlist requests as well as the escaped defects and we 
still need to develop the discipline to distinguish between those.

I personally thinks that this would be a very process to put in place, but 
putting this into place would delay this experiment substantially.

Maybe we should just compare the post-hoc analysis and the change in cycle 
time.
> 
> Do we want a quantitative measure of how long it takes to land code before
> and during this experiment?  The difference from MP creation to PQM
> submission should show a drop.  And we don't have to measure anything if
> we all feel a change is "the right thing to do".


That's a local optimization measurement. We should compare the average cycle 
time which is currently between 12-14 days. If this really removes a flow 
impediment, we should see more stuff done in less time, so the average cycle 
time should go down.

-- 
Francis J. Lacoste
francis.lacoste@xxxxxxxxxxxxx

Attachment: signature.asc
Description: This is a digitally signed message part.