dolfin team mailing list archive

[Anders Logg <logg@xxxxxxxxx>]

On Thu, Nov 10, 2011 at 11:15:55PM +0100, Martin Sandve Alnæs wrote:
> On 10 November 2011 22:50, Garth N. Wells <gnw20@xxxxxxxxx> wrote:
> >
> >
> > On 10 Nov 2011, at 20:29, Anders Logg <logg@xxxxxxxxx> wrote:
> >
> >> The problem appeared only on the buildbots (not locally) so they
> >> didn't show up until I pushed and I couldn't fix them without testing
> >> my fixes on the buildbots.
> >>
> >> Except for two small remaining bugs that I have now fixed that were
> >> part of the missing unit tests that should have been running but
> >> weren't. These could have been tested locally.
> >>
> >> I agree revert should be the policy if a new feature makes the
> >> buildbot break, but what about unit tests?
> >>
> >
> > Depends how quickly they can be fixed. Fine with me if fixed quickly, but best commented out  + bug report registered if the fix will take some time.
>
> Yes, that's my opinion as well. Quick fix is often fine, but IMHO
> quick < an hour.

I think up to say 4 hours is reasonable, but I'll try to stick to 1
hour.

> It's a matter of economy: one developer breaking the tests
> potentially disrupts the workflow of several other developers.

That's why we have the personal buildbots so the above never happens.
My personal buildbot has given a timeout 9 out of the 10 last times
I've used it so I haven't really been able to use it. And the last
time I used it, the timeout happened to be caused by a slow unit test
that I added... We are buying a new and faster machine to run some
buildbots so this should improve.

> The point is, that when some of the tests are failing,
> no matter the reason, the test suite is basically broken
> as a tool for me to check if my latest changes pass the tests.
> When the tests do not pass for a period of time, everyone
> is effectively blocked from working safely towards the
> trunk/release branch.
>
> And a number of tests still fail locally here, for example (several of
> each of these):
>
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "./test.py", line 158, in testWrongEval
>     f2, f3 = Expressions("sin(3.0*x[0])*sin(3.0*x[1])*sin(3.0*x[2])",
> NameError: global name 'Expressions' is not defined
>
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "./test.py", line 232, in testWrongSubClassing
>     self.assertRaises(RuntimeError, noMemberValues)
> AssertionError: RuntimeError not raised
>
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "./Assembler.py", line 295, in test_subdomain_assembly_form_2
>     self.assertAlmostEqual(assemble(a1, mesh=mesh), 1.0)
> AssertionError: 2.25 != 1.0 within 7 places

That's strange. Those are the ones I just fixed:

- test.py has been removed (an old test that was still lying around)
- Assembler.py has been fixed (fixed definition of boundaries in parallel)

And one of the buildbots is green (pressed force build after pushing
two hours ago).

We should really get the buildbots to do partial rebuild without
instant-clean. If it takes more than 2 hours to rebuild, that's also
disruptive to the workflow.

--
Anders