← Back to team overview

launchpad-dev team mailing list archive

Re: warning: we will soon have much noise in the test results...

 

On Thu, 2010-08-05 at 08:06 +1200, Robert Collins wrote:
> I'm still unconvinced that this is *actually* closer to how the code
> runs in production, based on jml's response earlier in this thread.

I landed a few changes because of this. I think everyone should be aware
of two discoveries that concerned me.

1. The while looking at one report on a naked object I saw a test
description disagreed with a test result. I was really puzzled. Really,
because I wrote all the code and I knew re-reading it that it was wrong.
There was a typo in the attr assignment. Login with the wrapped object
showed that that attr did not exist. Using the real attribute did make
the test work as I expected: items orders from 'hot' to 'cold'

2. While wrapping the milestone I saw factory.makeMilestone() rarely
made a valid milestone. A milestone must have a product or distro
series. No test with a distro milestone could render a milestone,
release, or distribution view. We did not have a test that exercised
these conditions with the factory made milestone, but lp engineers have
tried to help users be creating milestones using SQL, and these same
milestones definitely cause exceptions last year.

> With that in mind I'm entirely happy with disabling this check pending
> future discussion, particularly as it is causing developers to feel
> lots of friction. 

It certainly stole a day from me. I think the registry team have
branches that fixed or fix all the issues reported running registry
tests.

-- 
__Curtis C. Hovey_________
http://launchpad.net/

Attachment: signature.asc
Description: This is a digitally signed message part


Follow ups

References