openstack team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

The QA team had its weekly IRC meeting yesterday. Here is a summary of 
the action items and decisions coming out of the meeting.

* Online summary:

http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-24-17.00.html

* Status of the project

We've more than DOUBLED the number of tests in the tempest test suite in 
the past month. Thanks to many folks -- Chris Fattarsi, Ravikumar 
Venkatesan, Rajalakshmi Ganesan, Sapan Kona, David Kranz, Rohit Karajgi, 
Katherine Elliot, Unmesh Gurjar, Jose Idar, Sarad Patel, Daryl Walleck 
and others -- for the great contributions to Tempest.

Currently, there are 272 tests, with a number of skipped tests. We are 
working diligently to reduce the number of skipped tests (see more 
below) as bugs in the core projects get addressed.

The Tempest gate job, which runs Tempest against a devstack-installed VM 
on one of four node providers for every commit into the core project 
trunks, has had some difficulties with stability as of late, but I'm 
working with the CI team to try and address some things from the Tempest 
side to make the test runs faster.

* Action items

 - Ravikumar_hp to find good example blueprint for QA functional test plan
 - jaypipes to write draft email to ML about QA team working with 
developers on functional test plans for all new feature blueprints.
 - jaypipes to create Skip Captain duties wiki page and rotation

* Decisions made

To combat the ever-increasing number of skipped test methods due to bugs 
in core projects, we are going to have a "Skip Captain" rotation that 
will be responsible for tracking the upstream bugs that are causing 
skips in the tempest test suite, removing the skips when the bugs are 
fix committed in their respective upstream core project, and going over 
a quick status segment regarding the number of skips added and removed 
during each weekly IRC meeting.

* Blockers identified

Dan Prince fixed the bug in Nova regarding deleting flavors that was our 
only blocker (thx Dan!)

The smoke test branch and Swift test branch need another week of baking...

All the best,
-jay