openstack-qa-team team mailing list archive

[Yaniv Kaul <ykaul@xxxxxxxxxx>]

On 10/22/2012 05:33 PM, Jay Pipes wrote:
> Hi Sean :)
>
> Here's a quick recap:
>
> We agreed:
>
> * nosetests just isn't a good foundation for our work -- especially
> regarding performance/parallelism

Any proposed alternatives?
I am looking from time to time at 
https://code.google.com/p/robotframework/ and wondering if it allow 
faster test cases development
> * We need to produce good, updated documentation on what different
> categories of tests are -- smoke, fuzz, positive/negative, etc -- and
> put this up on the wiki

Perhaps a naming convention to the test case names, such as 
component_category_test_case?
For example:
compute_sanity_launchVM()
compute_negative_launchVM()

> * We need to produce a template example test case for Tempest that
> provides excellent code examples of how to create different tests in a
> best practice way -- I believe David Kranz is going to work on this first?
> * We need to get traceability matrixes done for the public APIs -- this
> involves making a wiki page (or even something generated) that lists the
> API calls and variations of those calls and whether or not they are
> tested in Tempest

Wouldn't code coverage report be better?
If you call FunctionX(param1, param2) - and you've called it with 2 
different param1 values and always the default in param2 - what does it 
mean, from coverage perspective?

Thanks,
Y.

> * I will start the traceability matrix stuff and publish for people to
> go and update
> * Antoni from HP is going to investigate using things in testtools and
> testrepository for handling module and package-level fixtures and
> removing some of the nose-based cruft
> * I will be working on the YAML file that describes a test environment
> so that different teams that use different deployment frameworks can use
> a commonly-agreed format to describe their envs to the CI system
> * A new member of Gigi's team (really sorry, I've forgotten your name!
> :( ) is going to look at the fuzz testing discussion from earlier and
> see about prototyping something together that would be used for negative
> and security/fuzz testing -- this would enable us to remove the static
> negative tests from Tempest's main test directories. For the record (and
> for the team member whose name I have forgotten, here is the relevant
> link: https://lists.launchpad.net/openstack-qa-team/msg00155.html and
> https://lists.launchpad.net/openstack-qa-team/msg00156.html)
>
> Best,
> -jay
>
> On 10/19/2012 02:47 PM, Sean Gallagher wrote:
> > Daryl,
> > I wasn't able to make the follow-up meeting. :/
> >
> > Can you/ someone send out a recap?
> >
> > David: you had a list of items. Can you post or share that somewhere?
> >
> > We discussed Blueprints for managing some of the planning work.
> >
> > What about higher level planning docs? Useful? Do we have any? Should we?
> >
> > Re Google Hangout next week, I'm interested.
> >
> > -sean