fenics team mailing list archive

[Johan Hake <johan.hake@xxxxxxxxx>]

On Monday July 19 2010 12:54:44 Kristian Oelgaard wrote:
> On 23 June 2010 15:35, Kristian Oelgaard <k.b.oelgaard@xxxxxxxxx> wrote:
> > On 23 June 2010 10:55, Kristian Oelgaard <k.b.oelgaard@xxxxxxxxx> wrote:
> >> On 22 June 2010 19:03, Johan Hake <johan.hake@xxxxxxxxx> wrote:
> >>> On Tuesday June 22 2010 08:28:37 Kristian Oelgaard wrote:
> >>>> I've started writing the programmer's reference for FEniCS.
> >>>> One of the features that we decided on was that doc-strings for
> >>>> PyDOLFIN should be written and maintained as part of the documentation
> >>>> project and then added to the dolfin module on import.
> >>>> 
> >>>> I thought about doing this in the following way:
> >>>> 
> >>>> 1) Create a pseudo module 'dolfin-doc' which is a copy of the classes
> >>>> and functions in the 'real' dolfin module only it contains no code at
> >>>> all, just doc-strings. (This approach will also make it easy to create
> >>>> a script to check if all functions are documented or if any docs are
> >>>> obsolete).
> >>> 
> >>> Sounds good. I first thought of a structure (other than a dummy class)
> >>> that just mimics the class hierarchy, but in some way that is what you
> >>> actually suggests and it is probably as easy as anything else.
> >>> 
> >>>> 2) Use the autodoc functionality of Sphinx to create parts of the
> >>>> documentation for functions and classes
> >>>> 
> >>>> 3) Manually add additional information (in the reST file) and links to
> >>>> other parts of the documentation like demos etc. This will not be
> >>>> available using help() in the Python interpreter.
> >>>> 
> >>>> 4) In the dolfin.__init__.py function import the 'dolfin-doc' module
> >>>> and copy the doc-strings from all classes and functions to the classes
> >>>> and functions in the real dolfin module as was suggested by Johan
> >>>> Hake.
> >>>> 
> >>>> The problem with this approach is that assigning to __doc__ is not
> >>>> permitted for objects of 'type' type.
> >>>> 
> >>> :(
> >>> 
> >>> I did not anticipate this. Not sure why this is. I have got the
> >>> impression that numpy get around this. They use numpydoc to
> >>> dynamically add their documentation. It makes heavy use of sphinx, but
> >>> I couldn't figure how they get around that __doc__ is read-only.
> >> 
> >> To me it looks like numpydoc is a Sphinx extension that translates the
> >> Numpy docstrings into something that Sphinx can understand, not the
> >> other way around which is what we want.
> >> 
> >> http://projects.scipy.org/numpy/browser/trunk/doc/sphinxext/README.txt
> >> 
> >> So I think our best bet is to proceed with your suggestions below.
> >> 
> >> Kristian
> >> 
> >>> While it might be cool to look into what NumPy have done, (they also
> >>> define a pseudo classes, which they populate with docstrings, (look
> >>> into phantom_import.py), and they also define some nice format for the
> >>> reST used in the docstrings), I suggest two things we can do:
> >>> 
> >>> 1) SWIG can generate docstrings. We do that allready using parsed
> >>> doxygen documentation. All of this is gathered in docstrings.i. I
> >>> suggest generating such a file from our documentation. We need to turn
> >>> of %feature("autodoc","1") in dolfin.i to get rid of the long and
> >>> sometimes faulty generated signatures.
> > 
> > I turns out that it's only the __doc__ of the class I can't assign to,
> > not the __doc__ of member functions (and regular functions).
> > A simpler solution (at least for me) is to parse the cpp.py module
> > once generated and substitute all docstrings of classes with the
> > docstrings from the dolfindoc module rather than creating the
> > docstrings.i file.
> > 
> > Then for the classes that we manually add we use the method you
> > described below, but only for class.__doc__ .
> > 
> > class Foo(object):
> >    __doc__ = dolfindoc.Foo.__doc__
> >    def bar(self):
> >        "this doc string will be substituted with the
> > dolfindoc.Foo.__dict__["bar"].__doc__."
> >        pass
> 
> Unfortunately it turned out I was too quick to make this conclusion.
> For instance,
> for the Swig generated class Mesh in the module cpp I can't assign to
> 
> Mesh.__doc__
> 
> because it is a 'type' object as pointed our earlier, I can assign to:
> 
> Mesh.__dict__["__init__"].__doc__
> 
> but not
> 
> Mesh.__dict__["size"].__doc__
> 
> along with practically all other member functions.
> 
> The reason is that Swig, after the class definitions does this:
> 
> Mesh.size = new_instancemethod(_cpp.Mesh_size,None,Mesh)
> 
> to which no assignment is possible.
> 'AttributeError: attribute '__doc__' of 'instancemethod' objects is
> not writable'
> The code in Mesh.size is already
> 
>     def size(self, *args):
>         return _cpp.Mesh_size(self, *args)
> 
> so why Swig overrides the method I don't know. If it didn't I could
> assign to the docstring.
> 
> Recall that my initial plan was to simply parse the cpp.py file and
> substitute the docstrings of all classes with what the pseudo module
> dolfindoc would define. However, that doesn't seem so practical
> anymore since I would need to also comment out all lines where
> instancemethods are being assigned to class members such that on
> import (in dolfin/__init__.py) I can loop classes and assign to
> function docstrings. This is of course possible but seems a bit
> awkward and I don't know what implications the commenting out of
> instancemethod assignment will have.
> As I see it, two solutions remain.
> 
> 1) Take the pseudo module dolfindoc from the FEniCS documentation and
> create a docstring.i generator. The script
> dolfin/dolfin/swig/generator.py should then try to import the
> dolfindoc module, if successful use that to create docstring.i,
> otherwise use the Doxy2SWIG generator.

I thought that you landed on this alternative. The point is to generate a 
dosctrings.i file from the FEniCS documentation. Then for the extended Python 
layer we can use a generated Python module to assign the docstrings:

  class VariationalProblem(...):
     ...
     __doc__ generated_docstring_module.VariationalProblem.__doc__

> One question w.r.t this approach, when is docstrings.i being
> generated? I see it is distributed with DOLFIN but shouldn't it be
> generated everytime DOLFIN is rebuild with enablePython, or does
> enableDocs have to be switched on as well?

It is generated by running dolfin/swig/generate.py. It takes quite a long time 
so I think it is good to pre-generate this in the distribution.

> 2) Extend our Python layer with all functions and classes that we want
> to use the dolfindoc docstrings for.
> These should just be empty and redirect calls to classes and functions
> of the cpp. In dolfin/mesh/mesh.py:
> 
> class Mesh(cpp.Mesh):
>     def size(self):
>         try:
>             import dolfindoc.mesh.mesh
>             __doc__ = dolfindoc.mesh.mesh.Mesh.__doc__
>         except:
>             __doc__ = cpp.Mesh.__doc__
>         return cpp.Mesh.size()
> 
> An additional benefit of this approach is that the module structure
> can be identical to what we have in the _real_ DOLFIN, not as it is
> now where everything is dumped in the dolfin.cpp module.
> I don't know how much overhead this will create, alternatively we can
> skip the try/except clause and simply have the documentation as a
> dependency, or not add the docstrings for memberfunctions and add them
> later on import as was the original idea.

This sounds cumbersome and in the example above will the try: except clause be 
called everytime size is called.
 
Johan

> Suggestions and comments are more than welcome!
> 
> Kristian
> 
> > then in dolfin/__init__.py we load the classes as we do now from the
> > dolfin module, and then iterate over all functions and member
> > functions and substitute docstrings from the dolfindoc module.
> > 
> > Kristian
> > 
> >>> 2) The added python classes and methods can be documented using your
> >>> suggested approach, but instead of adding the docstring after class
> >>> creation, do it during class (method or function) creation, a la:
> >>> 
> >>>  class Foo(object):
> >>>      __doc__ = docstrings.Foo.__doc__
> >>>      ...
> >>> 
> >>> where docstrings is the generated module containing the docstrings.
> >>> 
> >>> Johan
> >>> 
> >>>> In other words we can't assign to the __doc__ of
> >>>> 
> >>>> class Foo(object):
> >>>>     "Foo doc"
> >>>>     pass
> >>>> 
> >>>> Which is a new-style class and found in UFL and the SWIG code in
> >>>> DOLFIN.
> >>>> 
> >>>> It works fine for
> >>>> 
> >>>> def some_function(v):
> >>>>     "function doc"
> >>>>     return 2*v
> >>>> 
> >>>> and
> >>>> 
> >>>> class Bar:
> >>>>     "Bar doc"
> >>>>     pass
> >>>> 
> >>>> which is the old-style class often found in FFC.
> >>>> 
> >>>> Does anyone have a solution or comments to the above approach, or
> >>>> maybe we can do it in a completely different way.
> >>>> 
> >>>> I read about some workaround for the 'assign to __doc__' problem, but
> >>>> it doesn't seem that nice and it might be a problem to incorporate in
> >>>> the SWIG generated code?
> >>>> 
> >>>> http://stackoverflow.com/questions/71817/using-the-docstring-from-one-
> >>>> metho d-to-automatically-overwrite-that-of-another-me
> >>>> 
> >>>> 
> >>>> Kristian
> >>>> 
> >>>> _______________________________________________
> >>>> Mailing list: https://launchpad.net/~fenics
> >>>> Post to     : fenics@xxxxxxxxxxxxxxxxxxx
> >>>> Unsubscribe : https://launchpad.net/~fenics
> >>>> More help   : https://help.launchpad.net/ListHelp
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~fenics
> Post to     : fenics@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fenics
> More help   : https://help.launchpad.net/ListHelp