fenics team mailing list archive

[Johan Hake <johan.hake@xxxxxxxxx>]

On Wednesday June 23 2010 09:29:30 Kristian Oelgaard wrote:
> On 23 June 2010 17:57, Johan Hake <johan.hake@xxxxxxxxx> wrote:
> > On Wednesday June 23 2010 07:35:56 Kristian Oelgaard wrote:
> >> On 23 June 2010 10:55, Kristian Oelgaard <k.b.oelgaard@xxxxxxxxx> wrote:
> >> > On 22 June 2010 19:03, Johan Hake <johan.hake@xxxxxxxxx> wrote:
> >> >> On Tuesday June 22 2010 08:28:37 Kristian Oelgaard wrote:
> >> >>> I've started writing the programmer's reference for FEniCS.
> >> >>> One of the features that we decided on was that doc-strings for
> >> >>> PyDOLFIN should be written and maintained as part of the
> >> >>> documentation project and then added to the dolfin module on
> >> >>> import.
> >> >>> 
> >> >>> I thought about doing this in the following way:
> >> >>> 
> >> >>> 1) Create a pseudo module 'dolfin-doc' which is a copy of the
> >> >>> classes and functions in the 'real' dolfin module only it contains
> >> >>> no code at all, just doc-strings. (This approach will also make it
> >> >>> easy to create a script to check if all functions are documented or
> >> >>> if any docs are obsolete).
> >> >> 
> >> >> Sounds good. I first thought of a structure (other than a dummy
> >> >> class) that just mimics the class hierarchy, but in some way that is
> >> >> what you actually suggests and it is probably as easy as anything
> >> >> else.
> >> >> 
> >> >>> 2) Use the autodoc functionality of Sphinx to create parts of the
> >> >>> documentation for functions and classes
> >> >>> 
> >> >>> 3) Manually add additional information (in the reST file) and links
> >> >>> to other parts of the documentation like demos etc. This will not
> >> >>> be available using help() in the Python interpreter.
> >> >>> 
> >> >>> 4) In the dolfin.__init__.py function import the 'dolfin-doc' module
> >> >>> and copy the doc-strings from all classes and functions to the
> >> >>> classes and functions in the real dolfin module as was suggested by
> >> >>> Johan Hake.
> >> >>> 
> >> >>> The problem with this approach is that assigning to __doc__ is not
> >> >>> permitted for objects of 'type' type.
> >> >>> 
> >> >> :(
> >> >> 
> >> >> I did not anticipate this. Not sure why this is. I have got the
> >> >> impression that numpy get around this. They use numpydoc to
> >> >> dynamically add their documentation. It makes heavy use of sphinx,
> >> >> but I couldn't figure how they get around that __doc__ is read-only.
> >> > 
> >> > To me it looks like numpydoc is a Sphinx extension that translates the
> >> > Numpy docstrings into something that Sphinx can understand, not the
> >> > other way around which is what we want.
> >> > 
> >> > http://projects.scipy.org/numpy/browser/trunk/doc/sphinxext/README.txt
> >> > 
> >> > So I think our best bet is to proceed with your suggestions below.
> >> > 
> >> > Kristian
> >> > 
> >> >> While it might be cool to look into what NumPy have done, (they also
> >> >> define a pseudo classes, which they populate with docstrings, (look
> >> >> into phantom_import.py), and they also define some nice format for
> >> >> the reST used in the docstrings), I suggest two things we can do:
> >> >> 
> >> >> 1) SWIG can generate docstrings. We do that allready using parsed
> >> >> doxygen documentation. All of this is gathered in docstrings.i. I
> >> >> suggest generating such a file from our documentation. We need to
> >> >> turn of %feature("autodoc","1") in dolfin.i to get rid of the long
> >> >> and sometimes faulty generated signatures.
> >> 
> >> I turns out that it's only the __doc__ of the class I can't assign to,
> >> not the __doc__ of member functions (and regular functions).
> >> A simpler solution (at least for me) is to parse the cpp.py module
> >> once generated and substitute all docstrings of classes with the
> >> docstrings from the dolfindoc module rather than creating the
> >> docstrings.i file.
> >> 
> >> Then for the classes that we manually add we use the method you
> >> described below, but only for class.__doc__ .
> >> 
> >> class Foo(object):
> >>     __doc__ = dolfindoc.Foo.__doc__
> >>     def bar(self):
> >>         "this doc string will be substituted with the
> >> dolfindoc.Foo.__dict__["bar"].__doc__."
> >>         pass
> >> 
> >> then in dolfin/__init__.py we load the classes as we do now from the
> >> dolfin module, and then iterate over all functions and member
> >> functions and substitute docstrings from the dolfindoc module.
> > 
> > Sounds good to me. And remember that we only need to use the SWIG
> > docstring feature for classes that we do not extend in our own Python
> > layer.
> > 
> > Have we landed on a unified way of expressing parameters to methods,
> > functions and class initialization? NumPy relies on reST and they have
> > defined what should go into a docstring. Their example is quite
> > extensive and we should
> 
> > probably cut it down, but I think it is a nice reference:
> We have not yet decided on a format mainly because I didn't know what
> was possible and how we should go about things so suggestions are
> welcome.
> 
> >  http://projects.scipy.org/numpy/browser/trunk/doc/EXAMPLE_DOCSTRING.txt
> 
> This looks OK to me, also compare to:
> >>> import numpy
> >>> help(numpy.random.multivariate_normal)
> 
> and finally:
> 
> http://docs.scipy.org/doc/numpy-1.3.x/reference/generated/numpy.random.mult
> ivariate_normal.html#numpy.random.multivariate_normal
> 
> It looks like the string from help() is just the raw reST docstring,
> if we want to do this too we should definitely just put everything in
> the dolfindoc pseudo module and let Sphinx do its magic.

I think it looks good, and importantly not too complicated. I think the NumPy 
guys have put quite a lot of thoughts into this and it is good to not reinvent 
the wheel.

> Initially, I thought about mixing some manually written reST intput
> with some autodoc from Sphinx from the module docstring but I guess
> having everything available from
> 
> >>> help()
> 
> is a good idea?

I am not sure what that means, probably because I have not been able to follow 
your work closely.

> For the C++ interface we don't have to worry, we simply write
> everything manually in the reST files in the documentation source.
> (The two should, however, look very similar in structure in my opinion).

It would be nice to follow the docstring example from above for the C++ 
methods. We probably need to rearange the Parameters section to fit the 
different C++ types though.

Johan

> Kristian
> 
> > Johan
> > 
> >> Kristian
> >> 
> >> >> 2) The added python classes and methods can be documented using your
> >> >> suggested approach, but instead of adding the docstring after class
> >> >> creation, do it during class (method or function) creation, a la:
> >> >> 
> >> >>  class Foo(object):
> >> >>      __doc__ = docstrings.Foo.__doc__
> >> >>      ...
> >> >> 
> >> >> where docstrings is the generated module containing the docstrings.
> >> >> 
> >> >> Johan
> >> >> 
> >> >>> In other words we can't assign to the __doc__ of
> >> >>> 
> >> >>> class Foo(object):
> >> >>>     "Foo doc"
> >> >>>     pass
> >> >>> 
> >> >>> Which is a new-style class and found in UFL and the SWIG code in
> >> >>> DOLFIN.
> >> >>> 
> >> >>> It works fine for
> >> >>> 
> >> >>> def some_function(v):
> >> >>>     "function doc"
> >> >>>     return 2*v
> >> >>> 
> >> >>> and
> >> >>> 
> >> >>> class Bar:
> >> >>>     "Bar doc"
> >> >>>     pass
> >> >>> 
> >> >>> which is the old-style class often found in FFC.
> >> >>> 
> >> >>> Does anyone have a solution or comments to the above approach, or
> >> >>> maybe we can do it in a completely different way.
> >> >>> 
> >> >>> I read about some workaround for the 'assign to __doc__' problem,
> >> >>> but it doesn't seem that nice and it might be a problem to
> >> >>> incorporate in the SWIG generated code?
> >> >>> 
> >> >>> http://stackoverflow.com/questions/71817/using-the-docstring-from-on
> >> >>> e-m etho d-to-automatically-overwrite-that-of-another-me
> >> >>> 
> >> >>> 
> >> >>> Kristian
> >> >>> 
> >> >>> _______________________________________________
> >> >>> Mailing list: https://launchpad.net/~fenics
> >> >>> Post to     : fenics@xxxxxxxxxxxxxxxxxxx
> >> >>> Unsubscribe : https://launchpad.net/~fenics
> >> >>> More help   : https://help.launchpad.net/ListHelp
> >> 
> >> _______________________________________________
> >> Mailing list: https://launchpad.net/~fenics
> >> Post to     : fenics@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~fenics
> >> More help   : https://help.launchpad.net/ListHelp
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~fenics
> Post to     : fenics@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fenics
> More help   : https://help.launchpad.net/ListHelp