fenics team mailing list archive

[Kristian Ølgaard <k.b.oelgaard@xxxxxxxxx>]

On 7 September 2010 17:59, Johan Hake <johan.hake@xxxxxxxxx> wrote:
> [snip]
>
>> > But how do we extract the different arguments? I suppose this is
>> > collected by Doxygen, and we just need to parse these and output them in
>> > a correct way?
>>
>> I don't think we need to parse the arguments and output them. We just
>> get the function name and if we have more than one set of arguments
>> i.e., a different signature we know that we have an overloaded method
>> and how to handle it.
>
> And I guess the almighty generate_cpp_documentation.py script are able to
> extract the argument information?

No, but we should be able to figure this out from the signature (split
',' in '()').

>> The arguments should be described in the *Arguments* section of the
>> individual docstring with links to classes formatted like
>> _MeshEntity_, which we will substitute with :py:class:`MeshEntity` or
>>
>> :cpp:class:`MeshEntity` depending on which interface we document.
>
> Ok, but we only want that once for each method in python, even if it is
> overloaded?

No, I think we need to document the argument list for every overloaded
version like it is done in docstrings.dolfin.cpp.Mesh.

>> Although I just realized that standard C++ stuff like double* which
>> end up as numpy.array etc. should probably be handled.
>
> Yes this part I am a little worried about... But maybe a god handwritten
> lookup table will do the trick? At least for 99% of the cases ;)

I share your concern, but if, as you suggest, we'll be able to get God
to hand write our documentation I think we should be OK. :)
(a lookup table would be my best bet at the moment)

>> On a related note:
>> int some_func()
>> and
>> const int some_func() const
>> are different in C++, but in Python we don't have const right?
>> This will simplify the documentation a lot.
>
> Yes, we tend to %ignore all const versions of different methods.
>
> [snap]
>
>> >> >  * Extended methods needs to be handled in one of three ways:
>> >> >    1) Write the docstring directly into the foo_post.i file
>>
>> I like this option, if this is where we have the code for a function,
>> then this is where the docstring should be as it increases the
>> probability of the docstring being up to date.
>
> Ok, lets settle on this one. We also need to make sure that all %extended
> methods in the C++ layer gets a proper docstring. However I am not really sure
> how this can be done :P

I'm not sure I follow this, which %extended methods do you mean?

> [snup]
>
>> > Why do we need to assign to these methods? They already get their
>> > docstrings from the docstrings.i file. However if we want to get rid of
>> > the new_instancemethod assignment above, we can just remove the
>>
>> Some history.
>> Initially, we wanted to have all docstrings separated from the DOLFIN
>> code and collected in the fenics-doc module. The easiest way to get
>> the >>> help(dolfin) docstring correct is to assign to __doc__
>> dynamically.
>> If we could do this we wouldn't even need the docstrings.i file and
>> things would be simple.
>> However, we discovered that this was not possible, and because of that
>> we still need to generate the docstrings.i file.
>> Then, still assuming we wanted to separate docs from code and keeping
>> docstrings in fenics-doc, I thought it would be easier to generate the
>> docstrings.i file from the handwritten docstrings module in
>> fenics-doc.
>> Some methods don't get their docstrings from the docstrings.i file
>> though, so we still need to assign to __doc__ which is the easiest
>> thing to do.
>> Just recently we decided to extract the docstrings from the C++
>> implementation thus moving the docs back into DOLFIN. This makes the
>> docstrings module almost superfluous with the only practical usage is
>> to have documentation for the extended methods defined in the _post.i
>> files but if we put the docstrings directly in the _post.i files we no
>> longer need it.
>
> Ok, then I do not see any reason for a separate docstring module, makes life a
> lite bit easier...

Agree.

> [snep]
>
>> > I am confused. Do you suggest that we just document the extended Python
>> > layer directly in the python module as it is today? Why should we then
>> > dumpt the docstrings in a separate docstring module? So autodoc can have
>> > something to shew on? Couldn't autodoc just shew on the dolfin module
>> > directly?
>>
>> I'm confused too. :) I guess my head has not been properly reset
>> between the changes in documentation strategies.
>> The Sphinx autodoc can only handle one dolfin module, so we need to
>> either import the 'real' one or the docstrings dolfin module.
>> If we can completely remove the need for the docstrings module, then
>> we should of course include the 'real' one.
>
> Ok!
>
>> >> Then programmer's writing the Python
>> >> layer just need to document while they're coding, where they are
>> >> coding just like they do (or should anyways) for the C++ part.
>> >
>> > Still confused why we need a certain docstring module.
>>
>> Maybe we don't.
>>
>> >> >  2) for the extended Python layer in the cpp.py
>> >> >
>> >> > For the rest, and this will be the main part, we rely on parsed
>> >> > docstrings from the headers.
>> >> >
>> >> > The python programmers reference will then be generated based on the
>> >> > actual dolfin module using sphinx and autodoc.
>> >>
>> >> We could/should probably use either the dolfin module or the generated
>> >> docstring module to generate the relevant reST files. Although we
>> >> might need to run some cross-checks with the Doxygen xml to get the
>> >> correct file names where the classes are defined in DOLFIN such that
>> >> we retain the original DOLFIN source tree structure. Otherwise all our
>> >> documentation will end up in cpp.rst which I would hate to navigate
>> >> through as a user.
>> >
>> > This one got to technical for me. Do you say that there is no way to
>> > split the documentation into smaller parts without relying on the c++
>> > module/file structure?
>>
>> But how would you split it?
>
> I do not know. But then I do not know what the generation step can take as
> different inputs.

The write_python_documentation step should probably take the dolfin
module and the intermediate representation.

>> It makes sense to keep the classes Mesh
>> and MeshEntity in the mesh/ part of the documentation. Unfortunately,
>> Swig doesn't add info to the classes in the cpp.py module about where
>> they were originally defined. This is why we need to pair it with info
>> from the xml output.
>
> Ok, but say we keep all documentation in one module. If you are able to pair
> the different classes or functions with a module name, or file name you are
> able to create documentation which is structured after this hierarchy?

We need to figure out something, having everything in the cpp.py
module would create one big mess and it makes sense to follow the
DOLFIN C++ structure even for the Python interface.

>> >> I vote for using the generated docstrings module for the documentation
>> >> since it should contain all classes even if some HAS_* was not
>> >> switched on, which brings me to the last question, how do we handle
>> >> the case where some ifdefs result in classes not being generated in
>> >> cpp.py? They should still be documented of course.
>> >
>> > I think we are fine if the server that generate the documentation has all
>> > optional packages, so the online documentation is fully up to date.
>>
>> Maybe, but I think I saw somewhere that depending on the ifdefs some
>> names would be different and we need the documentation to be complete
>> regardless of the users installation.
>
> Yes, I think this is most relevant for the different la backends.

I think this is where I saw it, I had a quick look again but couldn't find it.

>> >> Another issue we need to handle is any example code in the C++ docs
>> >> which must be translated into Python syntax. Either automatically, or
>> >> by some looking up in a dictionary, but that brings us right back to
>> >> something < 100% automatic.
>> >
>> > Would it be possible to have just pointers to demos instead of example
>> > code. I know it is common to have example code in Python docstrings but
>> > I do not think it is equally common to have this in C++ header files.
>>
>> Since when did we care about common in FEniCS? :) I think small
>> input/output examples are good even for C++, look at the Mesh class
>> for instance.
>
> Ok, but put it in another way. It does look quite funny with a Python example
> in the C++ header.

The Python examples which are currently in the Mesh.h file are only
there because I wrote the docstring for the Python Mesh class first,
copy pasted and didn't have time to change the syntax. :)
We should have C++ syntax examples only in the *.h files of course.

But if we start putting these in their own lookup file we
> are back again to the separate file we just abandone. We could maybe add some
> more markup for c++ and Python examples directly in the header?

Yes, or if the examples are really simple we can translate them
automatically. I don't know which is the best solution.

> [snop] sorry ran out of vowels...
>
>> > Which should serve us well.
>>
>> OK, looks reasonable. So it might be possible to do this after all.
>
> Well, I think we have to let the almight script have a go first ;)

Agree. :)

Kristian

> Johan
>