← Back to team overview

openstack-qa-team team mailing list archive

Re: Tempest and functional test success definition

 

I think the discussion about whether SSHing crossed that line was more
about sshing to control nodes to do setup.

SSHing is pretty useful, you just need to be able to tweak it some when
you are doing it.  I'd suggest adding in the ability to ssh as a
configuration option, along with options for how long to try sshing before
fail.  Its nice to be able to run quick tests without sshing as well as
the more full tests that include ssh.

Gabe

On 2/5/12 11:37 PM, "Daryl Walleck" <daryl.walleck@xxxxxxxxxxxxx> wrote:

>Good question. We had a discussion awhile back on whether SSHing into an
>instance broke the line between black and white box testing. We had
>people on both sides, but the discussion was tabled for a bit while we
>dealt with other issues. It's definitely something I'd like to talk about
>again. I'm doing some SSHing in the dev branch that I work from and find
>it fairly useful.
>
>Daryl
>
>On Jan 31, 2012, at 10:07 AM, David Kranz wrote:
>
>> The discussion about Libvirt File Injection on the main list reminded
>>me that I am not clear on what we mean that a functional test passes.
>>For example, in one version of the stress test we are going to check in,
>>the test assigns a floating ip and ssh's to the created server to make
>>sure it is, by some definition, working. It looks like the Tempest test
>>of server creation passes if the API response says it did. Is this
>>adequate? Another example is that if we are testing that a volume is
>>attached to a server, don't we have to ssh to the server and make sure
>>that the volume is accessible? Perhaps these aspects are tested by the
>>nova unit tests but at first glance it did not seem so.
>> 
>> -David
>> 
>> -- 
>> Mailing list: https://launchpad.net/~openstack-qa-team
>> Post to     : openstack-qa-team@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack-qa-team
>> More help   : https://help.launchpad.net/ListHelp
>
>
>-- 
>Mailing list: https://launchpad.net/~openstack-qa-team
>Post to     : openstack-qa-team@xxxxxxxxxxxxxxxxxxx
>Unsubscribe : https://launchpad.net/~openstack-qa-team
>More help   : https://help.launchpad.net/ListHelp



References