| Thread Previous • Date Previous • Date Next • Thread Next |
On 13.9.2012 18:21, Jay Pipes wrote:
On 09/13/2012 06:43 AM, Sean Dague wrote:On 09/13/2012 12:42 AM, Vishvananda Ishaya wrote:Brian Waldon and I discussed this today. We came up with an idea which is to have the tempest run on stable/<last-release> during the dev cycle, and after the last milestone turn it on during the bugfixing phase. The potential disadvantage of this approach is that it could potentially make bugfixes a little harder to get in, but it might help make things more stable. Definitely worth a discussion at the summit.I think that would be a great idea. I'm still not convinced we need to fully give up on tempest running fully on normal commits. Post Folsom release / pre summit, I'm going to take some time to dive into the problem. The nose auto parallelizing isn't currently working, but some manual approaches might.
I think unittests may be relatively easily split between several Jenkins jobs just by specifying which packages to run. Following might be executed in different Jobs:
nose -w PATH_TO_TEMPEST tempest.tests.compute nose -w PATH_TO_TEMPEST tempest.tests.image nose -w PATH_TO_TEMPEST tempest.tests.network
The nose parallel issue: https://github.com/nose-devs/nose/issues/550 and another blocking issue: https://github.com/nose-devs/nose/issues/551 Frankly, I am seriously doubting whether nosetests is the right solution to our problems. I know nosetests has a lot of features but there are serious impedance mismatches between nosetests (and the underlying unittest[2] library) that make it unwieldy for full functional integration tests. Unfortunately, it is a serious investment in time to switch to something else, considering the whole Tempest codebase pretty much depends on nose/unittest. We are at the point, however, where we really do need to solve the parallelization problem -- the more tests we add, the longer tempest takes to run, and the longer tempest takes, the smaller the chance of the full run being the gate for core projects.
Have you guys considered DTests? https://github.com/klmitch/dtestThere is also Jenkins plugin that can distribute JUnit tests between several workers. It would be great to have something same for Python unittest/nose/whatever. Surely there would have to be some thinking done about how to handle class/module fixtures.
Personally, I'm torn between spending time on a rewrite and trying to decipher nose's often incomprehensible codebase and pushing fixes to the upstream library. -jay
| Thread Previous • Date Previous • Date Next • Thread Next |
