openstack team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

Thank you to all who attended the OpenStack design summit last week
and expressed their interest and support for the ongoing efforts
around quality assurance in OpenStack.

As a reminder, if you are interested in participating in the OpenStack
QA community, please do join the Launchpad team here:

https://launchpad.net/~openstack-qa-team

Here is a summary of the decisions reached at the summit, and an
overall action plan for the OpenStack QA team for the Essex release
cycle.

1) Weekly QA Meeting

There will be a weekly meeting on IRC Freenode.net #openstack-meeting,
at 12:00 EDT (16:00 UTC) to discuss progress on QA activities,
coordinate sometimes duplicated efforts, and to expose delays or
blockers that need attention.

2) Small/Unit Test Tracking

Folks at NTT and other contributing organizations have already begun a
massive analysis of the code coverage and test quality of the Nova
source code. Nati Ueno gave a session about the process they are using
to track these analyses and feed bugs back into the Nova project. Here
is a summary of this process:

A) Each Nova component's test quality and coverage is being tracked in
a bug report. The bug report is to belong to both the OpenStack QA
project AND to the upstream OpenStack core project (Nova, Glance, etc)

B) Linked to these bug reports is a traceability matrix, done using a
Google Doc Spreadsheet, that outlines the coverage and quality of
tests for all essential methods and functions in a module

C) Once analysis is completed, work begins on constructing new test
cases for missing coverage areas and refactoring weak existing test
cases. These tests are then proposed for merging into the core
project's trunk, and go through code review like all other branches.

3) Stabilization/Maintenance Branches

A group comprised of some of the QA team and other interested parties
will maintain Diablo stabilization/maintenance branches for Nova,
Glance, and possibly Keystone.

This group will monitor the Essex trunk source code and backport
critical bug fixes into these maintenance branches, cutting
maintenance releases when projects feel there have been enough
requests for such maintenance releases.

4) Integration Test Frameworks Being Combined

There was some excellent discussions around the current proliferation
of integration test suites and frameworks at the design summit. Some
big decisions around this area were made, and Gabe Westmaas will be
sending out an email specifically about these decisions in a few
moments.

The summary of these decisions is that we a) want to stop working on
multiple integration test frameworks and suites and focus our energy
on a single unified test project, and b) we have a good idea of the
differences between the existing suites and a plan to merge a lot of
the code.

Cheers, and please do respond if I got anything wrong above or you
wanted to add to the summary.
-jay