launchpad-dev team mailing list archive

[Bjorn Tillenius <bjorn@xxxxxxxxxxxxx>]

On Wed, Feb 10, 2010 at 09:17:36PM +0000, Gavin Panella wrote:
> Long doctests can accumulate large amounts of state. Object, test data
> and variable reuse causes headaches for developers and reviewers. This
> bothers me, so I brought it up at a reviewer's meeting a long long
> time ago:
> 
>   "Because of the accumulation of state in doctests I think it would
>    be good to limit new doctests to 200 lines or less. gmb wrote a new
>    doctest last week of just under 200 lines and it was still
>    perfectly comprehensible. That's where the number comes from."
> 
> The consensus was that a 200 line limit wasn't the answer, but there
> were lots of other ideas, so let's discuss it.

This is an excellent thing to discuss. In fact, this is one of the
things that is on my plate to deal with during this cycle, so hearing
what people think is interesting. I have some ideas on how to improve
the situate, and reading the IRC log, it looks like at least some
people. I don't know what the right thing to do is. I think this
requires some experimentation, testing a few different strategies and
see which one looks the best, and is most useful. My basic ideas so far
is to make the doctests into actual documentation. For each major part
of Launchpad, there should be a doctest describing the part. That
doctest can link to other doctests which describes the part in more
detail, or describes sub-parts. The doctest should be high-level, and
not describe too much detail, preventing them from breaking due to
trivial changes. More detailed tests could go into Python tests.

This is the basic idea that I'm going to experiment with first. It might
turn out it's not feasible. For example, good documentation is hard to
write, and it's only feasible to write good documentation if someone
actually reads it. So another challenge will be to actually make use of
the documentation, providing in a way that people will find it useful,
and actually read it. The less useful documentation proves to be, the
less we should spend time writing it.

That said, documentation can actually be useful for the one writing it.
This should not be under-estimated.  Explaining how things work, and
most importantly explaining why things work the way they do, forces you
to think about how easy things are to understand. If you have trouble
explaining it, maybe your design is not so good. I know that I have
realised this a few times, and I'm sure others would as well, if they
made the effort.


-- 
Björn Tillenius | https://launchpad.net/~bjornt