← Back to team overview

checkbox-dev team mailing list archive

Re: tests in old checkbox

 

On Fri, Sep 5, 2014 at 4:27 PM, Daniel Manrique
<daniel.manrique@xxxxxxxxxxxxx> wrote:
> I'm looking back at the tests we had in old checkbox :)
>
> We even had some tests for packaging and setup, but I don't think
> those would apply to plainbox.
>
> - Test that all the job (.txt.in) files are declared in setup.cfg

Correct me if I'm wrong but we don't have anything to do here. We
don't have setup.cfg's anymore

> - Test that all the job files are added to POTFILES.in

This is a bit harder to validate. I'll think about what to do to
address it. One thing I wanted to add was a automatic FileUnit so that
we can advise on the new file extension and deprecate .txt and
.txt.in. If this goes far enough we can actually generate POTFILES.in
ourselves and cut the need to validate but I'm not sure if we really
should always generate it (are there corner cases that we're missing
if we do)

> - Test that all the job files are included by local.txt.in (to avoid
> non-categorized jobs, plus checkbox wouldn't see them otherwise).

We'll start recommending category_id soon.

> For the message/job files themselves:
>
> - Test all jobs have name (id now)

Check and far more :-)

> - Test all jobs have either a command or a description (this may be
> obsolete now)

Check and far more :-)

> - Test all shell jobs have a description

Actually we check that all manual jobs have a description and
recommend having a description in general.

> - Test all shell jobs with user: root have a proper environ for any
> env variables referenced in their command:

We're currently not checking that. It's blocked by heredoc-aware
parser. I'll open a bug on that.
[https://bugs.launchpad.net/plainbox/+bug/1366192]

> - Test jobs comply with schema (it's to ensure they don't contain any
> unknown attributes but since plainbox is more free-form this is not
> needed now)

That's true but I think we will have some way to improve the
situation. First we cannot catch typos this way. We should still look
at all the unknown fields to see if they are fishy. Second, with the
provider unit we can statically declare which version of the de-facto
schema we're using and strictly validate that schema if it is known to
plainbox. If the schema refers to a more recent version than plainbox
(old copy of) supports we can disable this feature.

> - Test user-interact and user-verify jobs have a command

Check and more :-)

> - Test manual, user-verify and user-interact jobs have a parsable
> description (again, not needed since we don't do the
> PURPOSE/STEPS/VERIFICATION parsing stuff anymore).

We don't check that.

> A lot of them are obsolete now but it may be a good guideline on what
> to test for. Most of them were inspired by actual problems we
> encountered in the field, so the tests were added to save some time.

Thanks

Sending CC to the mailing list for reference
ZK