openstack-qa-team team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

On 09/13/2012 06:43 AM, Sean Dague wrote:
> On 09/13/2012 12:42 AM, Vishvananda Ishaya wrote:
>> Brian Waldon and I discussed this today. We came up with an idea which is to have the tempest run on stable/<last-release> during the dev cycle, and after the last milestone turn it on during the bugfixing phase. The potential disadvantage of this approach is that it could potentially make bugfixes a little harder to get in, but it might help make things more stable.
>>
>> Definitely worth a discussion at the summit.
> 
> I think that would be a great idea. I'm still not convinced we need to 
> fully give up on tempest running fully on normal commits. Post Folsom 
> release / pre summit, I'm going to take some time to dive into the 
> problem. The nose auto parallelizing isn't currently working, but some 
> manual approaches might.

The nose parallel issue:

https://github.com/nose-devs/nose/issues/550

and another blocking issue:

https://github.com/nose-devs/nose/issues/551

Frankly, I am seriously doubting whether nosetests is the right solution
to our problems. I know nosetests has a lot of features but there are
serious impedance mismatches between nosetests (and the underlying
unittest[2] library) that make it unwieldy for full functional
integration tests.

Unfortunately, it is a serious investment in time to switch to something
else, considering the whole Tempest codebase pretty much depends on
nose/unittest.

We are at the point, however, where we really do need to solve the
parallelization problem -- the more tests we add, the longer tempest
takes to run, and the longer tempest takes, the smaller the chance of
the full run being the gate for core projects.

Personally, I'm torn between spending time on a rewrite and trying to
decipher nose's often incomprehensible codebase and pushing fixes to the
upstream library.

-jay