openstack-qa-team team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

On 04/30/2012 09:24 AM, Karajgi, Rohit wrote:
> The coverage can be expanded quite a bit by having tests that perform certain 'white-box' actions such as updating DB entries, and restoring them at the end of the test.
> But most of these tests would be negative, but important, scenarios.
> These should be avoided if they can be replaced with RESTful APIs. However, if we do need to use them we need to give them some identifier so that they do not interfere
> with the normal execution of Tempest blackbox tests.

Yes, I've got some code to help in this regard coming up shortly. I'm 
waiting on Daryl's BaseComputeTest merge proposal to get into trunk 
before I propose -- and I was hoping to have a discussion this Thursday 
about this.

> Sure, we could avoid too many attrs within tempest so that we don't have to create different nose command sets with arguments for every attr combination. I'll check how
> we can get this done through configs.

Actually, nose's @attr is pretty good for this type of thing... though 
we could add a @class_attr decorator to allow entire tests to be decorated.

> Agreed with David that the default for tempest should not be devstack.

Tempest was designed to be entirely agnostic when it comes to the 
environment it is executing against. It shouldn't matter whether Tempest 
is running against Devstack, a Chef-deployed OpenStack cluster, or a 
cluster deployed manually with distro packages.

Tempest doesn't "default" to anything. It just so happens that the 
gating job that Jenkins runs does use devstack to spin up an OpenStack 
environment, but that doesn't really have anything to do with Tempest 
itself.

> As for my specific use case to cause expired tokens, I think Daryl's suggestion to update the Keystone API is great, since it is useful to have some kind of forced
> disable/invalidate of tokens.

Yes, agreed.

Best,
-jay

> Regards,
> Rohit
>
> -----Original Message-----
> From: openstack-qa-team-bounces+rohit.karajgi=nttdata.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-qa-team-bounces+rohit.karajgi=nttdata.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of David Kranz
> Sent: Monday, April 30, 2012 6:28 PM
> To: openstack-qa-team@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Openstack-qa-team] Devstack dependent tests
>
> We should have as much coverage as possible. But since we have been using config file variables to control which tests to run based on configuration (such as resize_available) it would be better to introduce a new config variable rather than making people tweak the arguments to nose. I also think the default for tempest should be that it is not using devstack.
>
>    -David
>
> On 4/30/2012 12:25 AM, Karajgi, Rohit wrote:
> > Hi,
> >
> > We are writing new tests for keystone and some of these tests need to touch keystone database.
> > I really want to avoid  this, but unfortunately there are no RESTful APIs supported in stable/essex to do the job.
> >
> > One of the example is
> > 1.	Check if get_tenants api fails for expired token. There is no way I can set expiry date of the token using admin RESTful API.
> >
> > So currently I'm planning to use mysql client commands and set the expiry date.
> > All such tests will be put in the attr decorator with the name "devstack". So any one who doesn't want to run such tests should run tempest with nosetests -a kind!=devstack.
> >
> > Does it make sense to add such tests?
> >
> > Regards,
> > Rohit
> >
> > ______________________________________________________________________
> > Disclaimer:This email and any attachments are sent in strictest
> > confidence for the sole use of the addressee and may contain legally
> > privileged, confidential, and proprietary data.  If you are not the
> > intended recipient, please advise the sender by replying promptly to
> > this email and then delete and destroy this email and any attachments
> > without any further use, copying or forwarding
>
>
> --
> Mailing list: https://launchpad.net/~openstack-qa-team
> Post to     : openstack-qa-team@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack-qa-team
> More help   : https://help.launchpad.net/ListHelp
>
> ______________________________________________________________________
> Disclaimer:This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data.  If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding