fenics team mailing list archive

[Kristian Ølgaard <k.b.oelgaard@xxxxxxxxx>]

On 27 August 2010 18:53, Anders Logg <logg@xxxxxxxxx> wrote:
> On Fri, Aug 27, 2010 at 01:43:17PM +0200, Kristian Ølgaard wrote:
>> On 27 August 2010 13:14, Anders Logg <logg@xxxxxxxxx> wrote:
>> > On Fri, Aug 27, 2010 at 01:09:06PM +0200, Kristian Ølgaard wrote:
>> >> On 27 August 2010 12:12, Kristian Ølgaard <k.b.oelgaard@xxxxxxxxx> wrote:
>> >> > On 27 August 2010 12:00, Garth N. Wells <gnw20@xxxxxxxxx> wrote:
>> >> >>
>> >> >>
>> >> >> On 27/08/10 10:51, Kristian Ølgaard wrote:
>> >> >>> On 27 August 2010 11:31, Garth N. Wells <gnw20@xxxxxxxxx> wrote:
>> >> >>>>
>> >> >>>>>>>>> The stuff that you have written for the Mesh class could easily go in
>> >> >>>>>>>>> to Mesh.h without causing too much clutter (reST looks nice), and I
>> >> >>>>>>>>> imagine it would be easy to add a folding mode to Emacs and other
>> >> >>>>>>>>> editors that will hide all lines starting with /// except for the
>> >> >>>>>>>>> first line.
>> >> >>>>>>>>>
>> >> >>>>>>>>> The simple script I wrote seems to work pretty well to extract the
>> >> >>>>>>>>> documentation. If it breaks somewhere, we could either improve the
>> >> >>>>>>>>> script or learn to write the code so the script does not break.
>> >> >>>>>>>>>
>> >> >>>>>>>>> The point here is that now the generated .rst files are in sync with
>> >> >>>>>>>>> the code, but in a day or two someone will edit one of the .h files in
>> >> >>>>>>>>> DOLFIN and the documentation and code will start to diverge.
>> >> >>>>>
>> >> >>>>> On second thought, what do you mean by diverge?
>> >> >>>>> I have test scripts in place the checks if a function in *.h is
>> >> >>>>> documented in *.rst, and if a function in *.rst is still present in
>> >> >>>>> *.h.
>> >> >>>>>
>> >> >>>>> If you mean the docstrings might change, we can perform the additional
>> >> >>>>> check where we test if the one liner docstring in *.h is present in
>> >> >>>>> the documentation in *.rst, then there can be no divergence and we can
>> >> >>>>> have short comments in the DOLFIN source code.
>> >> >>>>>
>> >> >>>>>>> Yes, but this problem is already there for the Python interface and it
>> >> >>>>>>> won't go away.
>> >> >>>>>>> I guess the key thing to this is that a new feature or a change in
>> >> >>>>>>> DOLFIN source code is not complete until the documentation has been
>> >> >>>>>>> updated.
>> >> >>>>>>>
>> >> >>>>>>
>> >> >>>>>> To save ourselves work for now, we could just let doxygen create the C++
>> >> >>>>>> programmers reference and provide a link to it. It doesn't seem very
>> >> >>>>>> sensible that we write our own parser to document the C++ code. With
>> >> >>>>>> doxygen, we also get class diagrams. We can then scan the doxygen
>> >> >>>>>> documentation for each class and improve it iteratively.
>> >> >>>>>
>> >> >>>>> Do you mean improve the Doxygen output, or the source  code (*.h
>> >> >>>>> files)? If we improve the output we can get diverging docs and code.
>> >> >>>>>
>> >> >>>>
>> >> >>>> I mean improve the strings following '///' in the .h files. In quite
>> >> >>>> some cases, just a few extra words would make a big difference.
>> >> >>>>
>> >> >>>> I'm coming around to putting all programming reference doc in the code.
>> >> >>>> I don't like lots of markup, but I don't see any other robust and easily
>> >> >>>> maintainable solution.
>> >> >>>
>> >> >>> As I wrote above, a test script is in place to pick up
>> >> >>> missing/obsolete docs, very little extra work is needed to also test
>> >> >>> if the short docstring in the source  code is correct. Then we run the
>> >> >>> tests as part of building the documentation.
>> >> >>>
>> >> >>
>> >> >> I just can't see myself hopping back and forth between the code and the
>> >> >> documentation when implementing and testing something new.
>> >> >
>> >> > I don't see why that would be necessary, the documentation can be
>> >> > updated and built later once the feature is in place and tested.
>> >> > But the feature can't be 'official' until it has been documented, it
>> >> > will require more self-discipline from the developers, which I don't
>> >> > think is necessarily a bad idea.
>> >> >
>> >> >>> I admit that the Doxygen output is much more detailed and the type
>> >> >>> information/links in argument lists is better compared to what is in
>> >> >>> Sphinx now, but that might change in the future (in Sphinx). On the
>> >> >>> downside, I personally find the Doxygen documentation overwhelming and
>> >> >>> I never use it for just that reason.
>> >> >>>
>> >> >>
>> >> >> Doxygen is an (imperfect) ready made solution for the programmers
>> >> >> reference - so one of my points is that we can forget about the C++
>> >> >> programmers reference for now and get on with the more importance task
>> >> >> of documenting demos. We can return to the C++ programmers reference
>> >> >> later (which, as you say, may improve in Sphinx in the future).
>> >> >
>> >> > I'm fine with using Doxygen and simply put a link to the index page, I
>> >> > just think it is worthwhile to carefully discuss the pros and cons.
>> >>
>> >> Let's wait for someone else (Anders?) to comment on this before making
>> >> a decision.
>> >
>> > I think we can have both side by side (since both are working), but I
>> > prefer having the reST comments as part of the code and using our own
>> > script to extract it (or in the future built-in support in Sphinx if
>> > that should come). If we use Doxygen, we will need to use some special
>> > Doxygen-markup.
>> >
>> >> On a related note, it is interesting that only the three of us appears
>> >> to have an opinion about the look and feel of the documentation of the
>> >> programmer's reference. Are we perhaps making a big deal out of
>> >> something which nobody will use anyway?
>> >
>> > I would be good to get some more comments from others.
>> >
>> >> > The Python interface still has to be documented manually since there
>> >> > is no way to extract docstrings from the source code since the
>> >> > intention is to add docstrings to the module.
>> >>
>> >> Since we do need this, I'll start documenting the Python interface
>> >> starting with the functions and classes in the cpp.py module.
>> >
>> > I still haven't understood completely how the Python documentation is
>> > organized, but won't we be documenting Python code (the stuff we write
>> > in site-packages/dolfin as part of the code?
>>
>> The idea was to simply use the docstrings from the pseudo docstrings
>> module since we need those anyway for everything which is in cpp.py.
>> At least that's what we agreed upon a long time ago.
>
> That might still be the best conclusion. The reason this is coming up
> again (although we discussed it at length before) is I understand more
> after having actually worked with the documentation over the last
> couple of days.
>
>> In short we document everything in the fenics-doc/docstrings/dolfin
>> module which has the exact same layout as the real dolfin module, but
>> it only contains docstrings (no code).
>
> Can we make this part of DOLFIN instead of copying it from the
> documentation repo? That would make it easier to keep in sync. So all
> documentation that goes into the programmer's reference (as part of
> the documentation) is in some way copied from DOLFIN to the
> documentation (not the opposite).

We can easily make the docstrings module part of DOLFIN, we just add
it to site-packages/dolfin/docstrings and write all documentation for
the Python interface in this module. This approach even makes more
sense compared to what we have now.
The only reason I put the docstrings module in fenics-doc is that we
initially wanted to separate the two completely, but I guess we have
found out it is not possible to do in a sensible way.

>> We use the docstrings module to create the docstrings.i file such that
>> we have the correct docstrings for the stuff in the dolfin/cpp.py
>> module because we can't assign to docstrings on import for certain
>> classes/functions.
>
> And this replaces the current system where the stuff in docstrings.i
> is extracted from C++? So all Python functions will have their own
> documentation and that is always unrelated to the C++ documentation?

Yes and yes. We will still need to generate the docstrings.i file
because some of the Swig output in cpp.py makes it impossible to
assign to certain docstrings dynamically (when importing dolfin). If
we have the docstrings module locally in DOLFIN we no longer have to
worry about where to find it when running the script
dolfin/swig/generate.py which is good.

> Wouldn't it be good to have the C++ extracted stuff and then overwrite
> it when necessary, to avoid duplication?

Maybe, but it is not straightforward because:
* overloaded functions in Python have to be dealt with in a special way.
* all Sphinx links to classes like :cpp:class`Foo` has to change to
:py:class:`Foo` (which is the simplest problem)
* the example code needs to be different.
* some other stuff that I didn't think of

Kristian

>> We also copy the docstrings module to site-packages/dolfin such that
>> we in e.g., dolfin/fem/form.py can do:
>>
>> import dolfin.docstrings
>>
>> class Form(cpp.Form):
>>    __doc__ = dolfin.docstrings.fem.Form.__doc__
>>
>> it might seem silly, but the big advantage is that we have all docs in
>> one place which was one of the arguments for splitting the
>> documentation into a different repository in the first place.
>
> ok.
>
> --
> Anders
>