Re: [Ayatana-dev] Manual tests: common location, format, and style

[Ted Gould <ted@xxxxxxxxxx>]

+1 Overall, a couple comments:

On Mon, 2011-11-07 at 18:30 +1300, Tim Penhey wrote:
> == Proposal ==
> 
> We have a directory in the root of any project called "manual-tests"
> 
> This directory can have subdirectories named anything sensible for the 
> project.  For example, unity may have: dash, launcher, panel, music-lens 
> et al.
> 
> In these directories there are plain text files, and here is the kicker:
>   * We use a standard wiki format (that has a simple parser),
>     like ReST, textile, markdown, creole
>     (I would say moin, but it doesn't have an easy to use stand-alone 
> parser).

I'm going to support ReST here because I think the parsers for the
format are already in Ubuntu Main, which makes our lives easier.

> The files group like tests.  Tests have:
>   * A unique name in an H1 style heading
>   * A paragraph explaining the test
>   * A set of steps to reproduce the test
>   * Expected output or result

We should probably note also if the test is the result of a bug being
filed.  i.e. to verify a bug fix.

> Tests that fail will have bugs filed.
> 
> The directories can then be trivially parsed by a nifty python script 
> and uploaded to a public site, probably under unity.ubuntu.com or 
> similar to make the tests easily accessible.

Also, I'm going to request that we have a common verifier for the
subformat we're creating so that we can automatically verify that the
manual tests match the proper format.

Oh, and a template generator, but I think I can write enough Python to
do that one :-)

		--Ted

Attachment: signature.asc
Description: This is a digitally signed message part