launchpad-dev team mailing list archive

[Robert Collins <robert.collins@xxxxxxxxxxxxx>]

A few thoughts here.

Firstly, am small note on the mechanics of landing stuff: Triggering
landings is not hard: hydrazine can do it via email to PQM, or via LP
merge queues (which work but there is uncertainty around their
robustness so not being used); LP merge queues should be usable via
tarmac in the same way. I - really - value the difference between
'good to land' and 'robot is landing it'. I deeply hope we don't lose
that *or* we change our culture surrounding 2-eyeballs-for-everything.
I mention this because its not clear to what configuration of Tarmac
is planned.

On Sun, Sep 26, 2010 at 10:02 AM, Maris Fogels
<maris.fogels@xxxxxxxxxxxxx> wrote:
> On 09/23/2010 11:53 PM, Maris Fogels wrote:
>> On 09/23/2010 12:19 AM, Martin Pool wrote:
>>> I just noticed today there's something like 40 approved ready-to-land
>>> reviews on <https://code.edge.launchpad.net/launchpad/+activereviews>,
>>> with about 10 over a month old.  It seems like a lot.... Why do you
>>> suppose they sit there for so long?
>>>
>>
>> Pure speculation here, but I think that it is partly a side-effect of
>> distributed development: you can not easily see where the work is, or where
>> undelivered work has built up.  We don't see or count the unmerged branches
>> every day (no one person does), and the system, people, and process do not warn
>> us when too much undelivered work has built up.
>>
>
> ...
>
>> You could double-up the Andon system to warn you about the amount of undelivered
>> code at each stage of development:
>>
>>   Unlanded: xxxxxxxxxxxx
>>   PQM:               xxx
>>   Buildbot:           xx
>>   QA:             xxxxxx
>>   Undeployed:   xxxxxxxx
>>
>>
>
> I just saw the flaw in my reasoning behind this idea.  First, I am going to
> share my mistake with the list so that others can avoid making the same error in
> the future.  Second, I have a proposal for dealing with the unlanded branches.
>
> The mistake I made is believing that making the built-up inventory visible will
> solve the problem: it won't.  Having a system that watches unlanded branches is
> no better than pointing a video camera at a stack of boxes: someone still has to
> get down there and move the stack; it is a safe bet that the video camera won't
> get up and help you.  The inventory builds up because there is no process or
> person in place to prevent it.  The cameras watching the pile will not stop it
> from growing.  We need process first.
>
> So I do not know how to keep the unlanded branches from building up, but I do
> have an idea about how we can get them landed without sending nag mails to everyone.
>
> My proposal is simple: we create a kanban card for every unmerged branch, then
> we put those cards in each respective team's "Ready To Land" lane.  The teams
> may then deal with the extra cards as they see fit.  This would create about 14
> new cards, all for code that is more than ten days old.  We won't worry about
> the extra cards blowing past the lane's maximum card limit - that is kind of the
> point :)
>
> Does anyone object to this idea, or see obvious flaws in it?  If not, I can put
> the plan into action on Monday.

The obvious flaw is that this is a misuse of kanban.

Kanban is meant to keep concurrent work low and reduce mental churn
and handoffs.

Moving a card out of acitve until it has been deployed is freeing up a
concurrent work slot that shouldn't be freed up : we haven't delivered
*anything* until it is in use.

The problem is that sending something to PQM is essentially a handoff;
I'd deal with this by:
 - increasing the concurrency figure in your lanes by 8 hours work
(one full test run with ec2 to find any missed tests, and room for
hiccups with testfix mode.
 - cards stay on your board until deployed.
 - you may want another lane for stuff that has been QA'd but
deployed, so that db-devel inventory doesn't make you hit WIP limits
unnecessarily.

-Rob