launchpad-dev team mailing list archive

[Maris Fogels <maris.fogels@xxxxxxxxxxxxx>]

On 05/10/2010 12:44 PM, Andrew Bennetts wrote:
> Maris Fogels wrote:
> > Hi all,
> >
> > Question for the room: should a test case leave behind OOPS reports
> > in /var/tmp/lperr.test?  Would detecting that a test left behind
> > OOPS reports when it was not supposed to increase the quality of our
> > code?
>
> My guess:
>
> No and yes, respectively.
>
> -Andrew.

Thanks Andrew.  Sounds good to me :)

So, a follow-up question then: would it be too much work to add such checking to 
the Launchpad test suite?

Here is the complete list of OOPS Exception-types I saw on the hung server: 
http://people.canonical.com/~mars/oopslist.txt

Looking through the list of OOPSes, there are a few head-scratchers, and a few 
that are entirely to be expected.  I suspect the 404 "Object Not Found" errors 
are fine to leave.  Some of these are blank Runtime errors, probably raised on 
purpose by tests (the tracebacks are missing: I have no way to tell).  I have no 
idea what to think about MemoryError OOPSes.

Checking for new OOPSes after a test is simple.  Figuring out which ones are 
expected is not.  How could we do this in a simple, unobtrusive way?

We could change TestCase.assertRaises() to do OOPS tracking.  Or the Zope 
testbrowser itself can track the errors it generates (404 errors mostly).  The 
programmer would get OOPS tracking for free.

In unit tests we could use a @decorator or TestCase property to signal that 
certain types of Exception generated by a unit test should be ignored.  We can 
do this on a test-by-test basis by writing:

  @expect_oops(RuntimeError)

That is about the most effortless way I can think of to flag something.

Everything not covered by the two cases above could raise a warning.

Sound sane?


Maris