openstack-qa-team team mailing list archive

[Daryl Walleck <daryl.walleck@xxxxxxxxxxxxx>]

I agree on the issue with the output from generated tests. That is troublesome, but from what I've seen in the source code, probably something that could be remedied. It's also very generous in it's parallel execution which is fine client-side, but can overwhelm a test environment since there's no configuration to throttle back the number of tests being executed at a time. Unfortunately I haven't seen a Python test runner that meets all the criteria that I'd like to have, thus this and other little proof of concepts I've been tossing around to see if any better approaches are out there.

Daryl
________________________________________
From: openstack-qa-team-bounces+daryl.walleck=rackspace.com@xxxxxxxxxxxxxxxxxxx [openstack-qa-team-bounces+daryl.walleck=rackspace.com@xxxxxxxxxxxxxxxxxxx] on behalf of Jaroslav Henner [jhenner@xxxxxxxxxx]
Sent: Monday, September 24, 2012 7:28 AM
To: openstack-qa-team@xxxxxxxxxxxxxxxxxxx
Subject: [Openstack-qa-team] PyVows proof of concept

In reply to:
https://lists.launchpad.net/openstack-qa-team/msg00236.html, which
didn't came to my mailbox for some reason (attachment?)

I tried pyVows myself. I kinda liked the concept, but I didn't like the
way it is reporting to JUnit format XML when using "generative testing":
http://heynemann.github.com/pyvows/#-using-generative-testing

In Jenkins, it looked like:

Test Result : Add
-----------------
should_be_numeric       0 ms    Passed
should_be_numeric       0 ms    Passed
should_be_numeric       0 ms    Passed
should_be_numeric       0 ms    Passed
should_be_numeric       0 ms    Passed
should_be_numeric       0 ms    Passed

The parameters to the testing method are important when using generative
testing, so I think they should be included in the name of the test. But
some funny characters like
()%* I don't remember which
are causing problems in Jenkins. I was investigating some problems with
them months ago with some other testing framework. I don't know how to
address this problem. It may be worthy to consider making some Robot
framework outputs plugin if generative testing is needed, or use Robot
Framework

https://wiki.jenkins-ci.org/display/JENKINS/Robot+Framework+Plugin

J.H.

--
Mailing list: https://launchpad.net/~openstack-qa-team
Post to     : openstack-qa-team@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack-qa-team
More help   : https://help.launchpad.net/ListHelp