openstack-qa-team team mailing list archive

[David Kranz <david.kranz@xxxxxxxxxx>]

I think this clearly illustrates why it is so important for QA to be 
done more in the open, and to post tempest blueprints for largish 
projects like this. I have a theory that one important reason so many 
organizations choose not to do this is that once they submit their tests 
to the public repository (in this case tempest), they have to get every 
change approved. I had the same issue when we submitted our stress 
tests. There cannot be any advantage to doing qa in private. We need to 
make submitting test code free of such barriers. More tests can only be 
good.

I propose that we set up a 'contrib' part of tempest that is exempt from 
core review.  At some point tests can be moved into tempest proper 
though, as a gating job, there is a limit to what can be there due to 
runtime considerations. But we already have a lot of tests running every 
night distributed across organizations. Why not post the code so others 
can share and improve it? We would need to:

1. Set up the contrib directory
2. Establish some guidelines for submitting changes to other people's 
contributions
3. Establish the nightly build for contributed tests, probably including 
running some outside of jenkins

I am not sure what review flexibility is afforded by gerrit that would 
help or hinder this effort.

Thoughts?

 -David


On 8/16/2012 3:40 PM, Dan Smith wrote:
> Hi Daryl,
>
> DW> Sorry for any confusion.
>
> I share at least half the blame here :/
>
> DW> I've had an implementation for this working for awhile, but failed
> DW> to port it in as I was working on getting other features
> DW> out. Regardless of the implementation, I'm really happy to see this
> DW> flexibility get intoTempest trunk. Since it seems you and your team
> DW> have already invested significant work into this, I'll end my effort
> DW> and focus on code reviews for the branches you've submitted.
>
> Okay, well, thanks. I guess I should have done some more comprehensive
> digging before I started down the path, so sorry for that.
>
> Thanks in advance for the reviews. I know what I've got probably isn't
> good enough to merge just yet, but am definitely looking forward to
> hearing from some tempestuous folks :)