launchpad-dev team mailing list archive

[Martin Pool <mbp@xxxxxxxxxxxxx>]

On 20 February 2010 02:14, Francis J. Lacoste
<francis.lacoste@xxxxxxxxxxxxx> wrote:
> On February 18, 2010, Martin Pool wrote:
>> If the bug is
>> clearly marked as "now live on edge, please test" then I think users
>> really will test it and tell you.
>
> How would they know that? Users' don't look at bugs unless when they have a
> problem... and even then!

My experience in open source is pretty much to the contrary of that:
if you ask people to test things, it's easy to do, and they see their
work has an impact, they will do it.

About half of Launchpad bugs (small random sample) are filed by
non-developers so clearly some people care about bugs.

It's true that commenting on the bugs will only notify people already
subscribed to the bugs.  However, the subscribers should be a set of
people who care about that particular bug and therefore want to see it
fixed properly.  This is a minority of users but an important one.

I think this enthusiasm is stymied at present because:

 * developers simply don't ask for feedback in most cases
 * it's not obvious where/how a feature can be tested because the
rollouts through staging/edge/lpnet are complicated and opaque; the
point where a bug is marked "fix released" does not mean you can test
it other than by running an instance yourself.

Think of a value stream map for the cycle:

 developer starts bug; developer sends patch; patch deployed; user
tests; user gives feedback; developer fixes fallout [repeat]

I think we can speed this by getting patches deployed for beta testing
sooner after they're written, by telling users when they can test it,
and by telling them in a way that they will test it as soon as they
get the mail and immediately give feedback.

Best of all is to say "please test this now", second best is "please
test this tomorrow", saying "please remember to test this in about 3
weeks" is very dull.

I think it would be a cheap experiment that, when closing a
user-facing bug developers say something like

 "This is fixed in trunk, will be on edge tomorrow and will be on
lpnet on the 22nd"

then you can see if people either say "no it's not fixed because... "
or if they file followon bugs about the feature.  If this is combined
with being able to have a second bite at features after they get beta
testing but before they're widely available, it may help a lot.

-- 
Martin <http://launchpad.net/~mbp/>