openstack-qa-team team mailing list archive

[Aaron Lee <aaron.lee@xxxxxxxxxxxxx>]

It looks like we have two different issues we are trying to solve here. The first is testing Nova itself, and the second is testing a wrapper to interface with Nova. I think the wrappers should be tested within their own projects. And as long as the wrapper is well understood enough it should be sufficient for testing Nova directly.

Daryl brings up a good point that we are going to have to worry about accessing Nove during the different phases of each test. If we don't trust the client enough to make the call we are testing, maybe we could build and call that url directly from the test. However maintaining these tests is going to be a huge pain(and time sink) if we expect everything to be set up by directly. In this case we could use a wrapper for setup, verification, and teardown, and call the tested action directly. If this is abstracted properly then we should only use httplib2 directly in the tests, and only use novaclient directly in the setup, teardown, and verification helpers.

Thanks for all the feedback,
Aaron

On Oct 17, 2011, at 4:16 AM, Brebner, Gavin wrote:

> David - my take is that we are doing QA on behalf of eventual customers; as such we cannot
> select any one API library for particular attention unless we are sure all customers
> will do the same. It's probably worth having a "blessed" library to guide customers to,
> and for QA to put significant effort on it, but we need to cover any way in which a 
> customer can legitimately access Nova, and need to work with the wider community to make
> any particular choice more than just something that's convenient for QA :) For functional/stress
> tests it's probably not such a bit deal as for API testing, but there are still enough 
> differences for me to be unhappy just to use one library.
> 
> My team have started largely using novaclient and euca2ools, but we also have folks looking
> into Java and Ruby libraries as we have customers using them, and also "raw" http calls - largely
> for API testing. I'd be happy to see python-novaclient become the "obvious" library for Nova customers 
> to use - but in the meantime I need to push my team to check out other libraries also.
> 
> This sound reasonable to folks ? Is anyone else seeing significant use of anything other than
> novaclient/euca2ools by customers ? 
> 
> 	Gavin 
> 
>> -----Original Message-----
>> From: openstack-qa-team-bounces+gavin.brebner=hp.com@xxxxxxxxxxxxxxxxxxx
>> [mailto:openstack-qa-team-bounces+gavin.brebner=hp.com@xxxxxxxxxxxxxxxxxxx] On
>> Behalf Of David Kranz
>> Sent: Friday, October 14, 2011 5:11 PM
>> To: openstack-qa-team@xxxxxxxxxxxxxxxxxxx
>> Subject: [Openstack-qa-team] Python wrappers for OpenStack APIs
>> 
>> I have been implementing some functional stress tests for nova.  It
>> seems that there are various pieces of Python code that have been
>> written to call the HTTP apis including novaclient as well as code that
>> is part of various test suites. There are also ueca-tools. It seems like
>> it would be useful to have a "standard" set of test driver code going
>> forward. Does any one have any comments about this, or a suggestion
>> about which client API code would be best to use?
>> 
>>  -David
>> 
>> 
>> 
>> --
>> Mailing list: https://launchpad.net/~openstack-qa-team
>> Post to     : openstack-qa-team@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack-qa-team
>> More help   : https://help.launchpad.net/ListHelp
> 
> -- 
> Mailing list: https://launchpad.net/~openstack-qa-team
> Post to     : openstack-qa-team@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack-qa-team
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, please delete it.