← Back to team overview

launchpad-dev team mailing list archive

Notes from Ubuntu discussion on QA metrics

 

Hello everyone,

One of the more enjoyable sessions I went to at the recent UDS was one
about getting good metrics for doing Ubuntu QA and release management.

There are no actions from this email, just sending in case you are as
interested as I am and would like to talk about it further.

You can find the blueprint here:
https://blueprints.edge.launchpad.net/ubuntu/+spec/ubuntutheproject-launchpad-n-metrichooks

I have taken the liberty to pasting the text for the link averse.
Following those are my own notes.

* * * *

Data rich, but we're not able to mine project databases efficiently

Selection Needs
 * Getting launchpad data using date restrictions and ranges (needed
for trend analysis)
 * Apply logical operation on multiple fields to pull out data ( and/or/not )

Input Needs
 * Work items and the whiteboard 'hack': Tracking work items in a better way

Display Needs
 * Output formats from Launchpad beyond WIKI
    * csv generation
    * field column selection to display
    * filter ontop of sets of output
 * Generating reports directly from Launchpad?
 * https://wiki.ubuntu.com/ReleaseFutureMetrics

Survey of what's active:
- Harvest
- QA Bug Dashboard:
- Weather Report:
- Burndown charts ( burn up varients, ...)
- Arsenal:
- Linaro reports: http://wiki.linaro.org/JamieBennett/MeetingNotes/2010-09-16

Other projects to consider leveraging:
Jamie Bennett's: The open source reporting framework, BiRT (Eclipse
plug-in), seems perfect for our reporting needs if we are to use an
external tool. I've successfully used the database from
people.canonical.com/~pitti/workitems/maverick/ to produce a simple
form listing all blueprints. BiRT can be used to generate dynamic
charts and also has a web-based aspect to produce dashboard like
interfaces for status monitoring.

BiRT:
http://www.eclipse.org/birt/phoenix/

Dynamic Web based Dashboard examples:
http://www.birt-exchange.com/be/demos/actuate-11/

* * * *

 * Want queries over date ranges so that we can do trend analysis
   * Currently using mail archives to do this. Blech
 * Want logical ops in search (and, or, not, parens, etc.)
 * Want cycle-based views and queries
 * Open up 'o' and 'p' series ASAP
 * Look into Eclipse BIRT

jml