launchpad-dev team mailing list archive

[Julian Edwards <julian.edwards@xxxxxxxxxxxxx>]

On Thursday 11 February 2010 09:44:25 Max Bowsher wrote:
> I've prepared a branch:
> 
> https://code.edge.launchpad.net/~maxb/launchpad/ignored-asserts
> 
> which fixes some assert statements which were inadvertently having no
> effect. It fails tests, because an assert which apparently should have
> been triggering all along now triggers.
> 
> The problem is the test of initialize-from-parent.py, which is used to
> test initialization of a new series parented from ubuntu/hoary - except
> apparently the sampledata ubuntu/hoary has pending builds, which fails
> the assert.
> 
> 
> 1) Is there anyone who knows about initialize-from-parent.py and could
> suggest an alternate series with pending builds that could be used in
> this test without otherwise sacrificing test coverage?
> 
> 2) If not, can anyone advise me how best to familiarize myself with the
> existing Soyuz sampledata, such that I could attempt to fix the test?
> 
> 
> Thanks,
> Max.
> 


Hi Max

Our sample data is horribly broken and we've known that for a few years.  The 
problem is that fixing it breaks tests, as you've discovered!  It's not even 
breaking them in a good way i.e. a genuine failure, it's side-effect testing 
that is not important to the thing being tested.

We're no longer worrying about this - it's simply too much work for too little 
benefit.  The way forward is to eschew the sample data in tests and make each 
test set up all the data it needs.  This will lead to much more focused tests 
that run faster and are easier to maintain.

If your branch is just cleaning up assertions, I'd definitely recommend 
looking into setting up new distroseries in the test.  It's a lot of work, but 
I can assure you it's far less work than fixing sample data, which *will* 
break a million other tests.  I know because I've been there :/

Oh and for the record, it's not Soyuz sample data, it's technically Registry 
:)

Cheers
J