launchpad-dev team mailing list archive

[Martin Pool <mbp@xxxxxxxxxxxxx>]

Thanks, that's really interesting to see.

Incidentally to get the editable form (again if you are at
canonical.com) you must actually use
<https://docs.google.com/a/canonical.com/document/d/1GNgTwk62WzG9oIN91bTZI4fNwfylYiSdXC56y9i_riQ/edit?hl=en_US>
otherwise you are redirected to a new empty document.

> 28% of bugs would have been prevented by proper testing

I guess this is almost guaranteed to be true for bugs that aren't
deployment, load, etc related, that you could possibly have written a
test to catch them.  I am interested in what would count as 'proper',
ie the spectrum between

- it wasn't tested at all, through to
- some cases were tested but others weren't and they failed, to
- in theory you could have written a test but it would have required
unusual foresight

Martin




On 22 October 2011 09:01, Francis J. Lacoste
<francis.lacoste@xxxxxxxxxxxxx> wrote:
> Hello launchpadders,
>
> As most of you are aware, I've been working on an analysis of our new
> critical bugs for a while now. (Seems like I started this at the end of
> August.) Anyway, I'm done collecting all the data and I have a draft
> analysis.
>
> I'm solliciting review of both the collected data, as well as the
> analysis and recommendations.
>
> The analysis is in a Google document, you can edit and leave comments in it.
>
> https://docs.google.com/a/canonical.com/document/d/1GNgTwk62WzG9oIN91bTZI4fNwfylYiSdXC56y9i_riQ
>
> The document is only accessible to Canonical employees, but there is a
> published version of the document available at
>
> https://docs.google.com/document/pub?id=1GNgTwk62WzG9oIN91bTZI4fNwfylYiSdXC56y9i_riQ
>
> You won't be able to comment inline there, but feel free to follow-up on
> the list.
>
> The actual data (in a spreadsheet) is linked from the analysis document.
>
> I'm joining a PDF version of the document, in case, anyone want to read
> it offline.
>
> tl;dr
>
> * Most of the new bugs (68%) are actually legacy issues lurking in our
> code base.
> * Performance and spotty test coverage represents together more than 50%
> of the cause of our new bugs. We should refocus maintenance on tackling
> performance problems, that's what is going to bring us the most bang for
> the bucks (even though it's not cheap).
> * As a team, we should increase our awareness of testing techniques and
> testing coverage. Always do TDD, maybe investigate ATDD to increase the
> coverage and documentation our the business rules we should be supporting.
> * We also need to pay more attention to how code is deployed, it's now
> very usual for scripts to be interrupted, and for the new and ancient
> version of the code to operate in parallel.
>
> --
> Francis J. Lacoste
> francis.lacoste@xxxxxxxxxxxxx
>
> _______________________________________________
> Mailing list: https://launchpad.net/~launchpad-dev
> Post to     : launchpad-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~launchpad-dev
> More help   : https://help.launchpad.net/ListHelp
>
>