launchpad-dev team mailing list archive

[Bjorn Tillenius <bjorn@xxxxxxxxxxxxx>]

When 3.0 is out of the way, the next priority will be to get the
currently failing Windmill test suite to pass.

I have made some changes to the way Windmill tests are run, but old
tests need to be converted in order to have them run in the new way. So
the way you will make sure that a Windmill test passes will be:

    1. Convert the test to a TestCase test
    2. Run 'make jscheck'

'make jscheck' runs only the new-style Windmill tests, and that test
suite currently passes. There's a buildbot running 'make jscheck'
daily:

    https://lpbuildbot.canonical.com/builders/jscheck

The old Windmill tests are run using the functest test runner. We want
to get rid of that test runner, and instead use our own bin/test test
runner. In order to do this, we need to convert the old tests to be
layered TestCase tests.

You can see examples of a new-style Windmill tests in
lib/lp/bugs/windmill/tests/test_bug_commenting.py and
lib/lp/code/windmill/tests/test_branch_links.py

The process of converting the tests is basically:

    1. Make sure that YourApplicationWindmillLayer exists.
       If not, create one in lp.yourapplication.windmill.testing. See
       lp.bugs.windmill.testing.py for an example.

    2. Make sure that bin/test --layer=YourApplicationWindmillLayer is
       run by 'make jscheck' (Check in Makefile)

    3. Move the python file to lp.yourapplication.windmill.tests. The
       test_foo.py file has to be directly under a tests/ directory,
       otherwise it won't be picked up.

    4. Create a subclass of TestCaseWithFactory, setting
       layer = YourApplicationWindmillLayer.

    5. Create a test_foo() method, moving the contents of the old
       Windmill test function inside it.

And that's it really. You can now run the test using bin/test
--layer=YourApplicationWindmillLayer -t test_foo. Specifying the layer
is important, since by default bin/test won't run tests in a
WindmillLayer.

As a bonus, you can now use the LaunchpadObjectFactory to do test setup.
Don't forget to commit transaction after the setup, since otherwise the
app server won't see the changes.

The db also gets reset between tests, so you don't have to clean up
after yourselves in the test.

Please let me know if you run into any problems.

Known issues:

    * If don't create a new branch, you might have to re-generate
      bin/test, since there were changes in test.in, and bin/test
      doesn't get re-genereated automatically. Removing bin/buildout and
      running 'make' fixes it. The symptom here is that bin/test
      --layer=WindmillLayer will run all the tests.

    * Running 'bin/test --layer=WindmillLayer' or running two tests in
      different WindmillLayer Hangs when the second layer is set up.

    * Sometimes an exception is raised (complaining about invalid thread
      ids) during teardown.

I have submitted fixes for the two latter issues upstream, and hopefully
it will be included in Windmill 1.3, so it will be fixes when we
upgrade. If it causes too much of a problem, I'll look into patching our
version locally.


-- 
Björn Tillenius | https://launchpad.net/~bjornt