openstack-qa-team team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

On 05/29/2012 07:28 PM, Daryl Walleck wrote:
> Well, if by complete failure you mean not configured, you're correct.
> :-) The cleanup solution has several parallel solutions in progress, and
> should be a coding standard going forward.

Perhaps, but I haven't seen any of those...

> The quotas issue is a matter
> of configuration the devstack environment. This is a matter of if we
> should expect Tempest to pass out of the box with any Devstack
> configuration, which may not be realistic. I know there was an issue not
> too far back where some tests related to volumes would not pass with the
> default Devstack config because there wasn't enough space allocated for
> volumes. However, I think we should be able to make some sensible
> default suggestions that should run in most "basic" environments.

The quotas != rate-limiting. Default quotas for Compute API resources 
came from Rackspace Cloud Servers, IIRC, and so they represent a "basic 
environment" pretty well, I'd say. That said, I do think that setting 
quotas higher for users used in tempest makes sense.

> At the development conference we had a discussion about how best to
> proceed with parallel execution. The general consensus was:
>
>   * Switching to py.test isn't a popular idea
>   * Folks would rather improve nose's parallel capabilities than develop
>     another solution
>
> I haven't tinkered with nose in the last few weeks, but I think it may
> be possible to simply run the Tempest with py.test without any
> modifications. This still wouldn't be a popular solution regardless, so
> let's go back to the problem we are trying to solve. I think we can all
> agree that we would like a series of decisive smoke tests that run in
> what is perceived to be a reasonable amount of time. So what is the bar
> we're trying to reach? My personal feelings is that the 10-15 minute
> range is more than reasonable using a full Linux distro, which means
> with Cirros we should be able to squeak by under the 10 minute mark.

I'd agree with that, yep. Unfortunately, Tempest is taking more than 
3200 seconds to complete now on some of the node providers. :(

> What's everyone else's feelings on smoke test pain point times? I think
> getting a sense of that will make any further decisions a bit more clear.

Regardless of what length of time we say, I still think we need to get 
parallel execution working properly (with the existing --processes 
option in nosetest's multiprocessing plugin).

best,
-jay