launchpad-dev team mailing list archive

[Michael Hudson <michael.hudson@xxxxxxxxxxxxx>]

On 25/03/10 01:22, noreply@xxxxxxxxxxxxx wrote:
> Merge authors:
>    Leonard Richardson (leonardr)
> ------------------------------------------------------------
> revno: 10575 [merge]
> committer: Launchpad Patch Queue Manager<launchpad@xxxxxxxxxxxxxxxxx>
> branch nick: launchpad
> timestamp: Wed 2010-03-24 12:18:57 +0000
> message:
>    [r=gary][ui=none] Make it possible to use launchpadlib in web service
>    	pagetests.

So this broke everything, basically.  And surprisingly.

This is from setupGlobs, the function that sets up globals for every 
page test, called before each page test is run.

@@ -725,6 +727,9 @@
         'launchpad-library', 'nopriv-read-nonprivate')
     test.globs['anon_webservice'] = LaunchpadWebServiceCaller(
         'launchpad-library', '')
+    test.globs['launchpad'] = Launchpad.login(
+        'launchpad-library', 'salgado-change-anything', 'test',
+        service_root='http://api.launchpad.dev/', version="devel")
     test.globs['setupBrowser'] = setupBrowser
     test.globs['setupDTCBrowser'] = setupDTCBrowser
     test.globs['setupRosettaExpertBrowser'] = setupRosettaExpertBrowser

There are three main problems:

1) It's probably not sane to do something as expensive as logging in to
launchpad before every page test.

2) Having the default Launchpad object be logged in as a sample data 
(bad) admin (double plus bad!) is not very sane either.  The webservice 
object also has this issue.

3) (this is the novel and exciting bit) For reasons that are not 
entirely clear[1] it causes running 
test_dir_construction_and_trivial_running from 
canonical.launchpad.testing.tests.test_pages.TestMakeStoryTest to kill 
the test suite run:

Running canonical.testing.layers.PageTestLayer tests:
  Set up canonical.testing.layers.BaseLayer in 0.004 seconds.
  Set up canonical.testing.layers.DatabaseLayer in 0.453 seconds.
  Set up canonical.testing.layers.LibrarianLayer in 9.127 seconds.
  Set up canonical.testing.layers.MemcachedLayer in 0.122 seconds.
  Set up canonical.testing.layers.LaunchpadLayer in 0.000 seconds.
  Set up canonical.testing.layers.FunctionalLayer in 6.323 seconds.
  Set up canonical.testing.layers.LaunchpadFunctionalLayer in 0.000 seconds.
  Set up canonical.testing.layers.GoogleServiceLayer in 2.632 seconds.
  Set up canonical.testing.layers.PageTestLayer in 0.001 seconds.
  Running:
 test_dir_construction_and_trivial_running (canonical.launchpad.testing.tests.test_pages.TestMakeStoryTest)Traceback (most recent call last):
  File "./bin/test", line 259, in <module>
    result = testrunner.run([])
  File "/home/mwh/canonical/checkouts/trunk/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/__init__.py", line 32, in run
    failed = run_internal(defaults, args, script_parts=script_parts)
  File "/home/mwh/canonical/checkouts/trunk/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/__init__.py", line 45, in run_internal
    runner.run()
  File "/home/mwh/canonical/checkouts/trunk/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/runner.py", line 136, in run
    self.run_tests()
  File "/home/mwh/canonical/checkouts/trunk/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/runner.py", line 216, in run_tests
    setup_layers, self.failures, self.errors)
  File "/home/mwh/canonical/checkouts/trunk/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/runner.py", line 374, in run_layer
    return run_tests(options, tests, layer_name, failures, errors)
  File "/home/mwh/canonical/checkouts/trunk/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/runner.py", line 306, in run_tests
    test(result)
  File "/usr/lib/python2.5/unittest.py", line 281, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python2.5/unittest.py", line 278, in run
    result.stopTest(self)
  File "/home/mwh/canonical/checkouts/trunk/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/runner.py", line 724, in stopTest
    self.testTearDown()
  File "/home/mwh/canonical/checkouts/trunk/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/runner.py", line 680, in testTearDown
    layer.testTearDown()
  File "/home/mwh/canonical/checkouts/trunk/lib/canonical/testing/profiled.py", line 29, in profiled_func
    return func(cls, *args, **kw)
  File "/home/mwh/canonical/checkouts/trunk/lib/canonical/testing/layers.py", line 728, in testTearDown
    DatabaseLayer.connect().close()
  File "/home/mwh/canonical/checkouts/trunk/lib/canonical/testing/profiled.py", line 29, in profiled_func
    return func(cls, *args, **kw)
  File "/home/mwh/canonical/checkouts/trunk/lib/canonical/testing/layers.py", line 804, in connect
    return LaunchpadTestSetup().connect()
  File "/home/mwh/canonical/checkouts/trunk/lib/canonical/ftests/pgsql.py", line 270, in connect
    self._connectionString(self.dbname, self.dbuser)
  File "/home/mwh/canonical/checkouts/trunk/lib/canonical/ftests/pgsql.py", line 123, in fake_connect
    return ConnectionWrapper(_org_connect(*args, **kw))
psycopg2.OperationalError: FATAL:  database "launchpad_ftest" does not exist

4) Then, because of a bug in zope.testing[2], this causes the thread 
monitoring the subprocess running these tests (or something like that) 
to error out without reporting failure.  So any errors in tests that 
run in the same subprocess as test_dir_construction_and_trivial_running 
-- and there are 7000 odd of them! -- are not found by running the test 
suite in the usual way, even if the errors occur before the subprocess 
craps out.  Two fail before this point currently.

So in summary, we have 7000 tests that could be failing currently.  Not
ideal in release week.  I have a branch that comments out the 
instantiation of the Launchpad and am running it though ec2 to see if 
any more of these tests are in fact failing.  I'll land it after I've 
fixed any such failures.

Thanks to wgrant for pointing out the failing but not-failing tests, and
finding which exact hunk of which commit was causing the failure.

Cheers,
mwh

[1] It seems that the only reason test_dir_construction_and_trivial_running
    doesn't fail already is that the page tests it runs are in fact so 
    trivial they don't make any database connections -- changing them so
    they do causes a failure like the one above.

[2] This bug is fixed in the new zope.testing that jml is trying to
    land.  Hurrah!