fenics team mailing list archive

[Kristian Oelgaard <k.b.oelgaard@xxxxxxxxx>]

On 23 June 2010 15:35, Kristian Oelgaard <k.b.oelgaard@xxxxxxxxx> wrote:
> On 23 June 2010 10:55, Kristian Oelgaard <k.b.oelgaard@xxxxxxxxx> wrote:
>> On 22 June 2010 19:03, Johan Hake <johan.hake@xxxxxxxxx> wrote:
>>> On Tuesday June 22 2010 08:28:37 Kristian Oelgaard wrote:
>>>> I've started writing the programmer's reference for FEniCS.
>>>> One of the features that we decided on was that doc-strings for
>>>> PyDOLFIN should be written and maintained as part of the documentation
>>>> project and then added to the dolfin module on import.
>>>>
>>>> I thought about doing this in the following way:
>>>>
>>>> 1) Create a pseudo module 'dolfin-doc' which is a copy of the classes
>>>> and functions in the 'real' dolfin module only it contains no code at
>>>> all, just doc-strings. (This approach will also make it easy to create
>>>> a script to check if all functions are documented or if any docs are
>>>> obsolete).
>>>
>>> Sounds good. I first thought of a structure (other than a dummy class) that
>>> just mimics the class hierarchy, but in some way that is what you actually
>>> suggests and it is probably as easy as anything else.
>>>
>>>> 2) Use the autodoc functionality of Sphinx to create parts of the
>>>> documentation for functions and classes
>>>>
>>>> 3) Manually add additional information (in the reST file) and links to
>>>> other parts of the documentation like demos etc. This will not be
>>>> available using help() in the Python interpreter.
>>>>
>>>> 4) In the dolfin.__init__.py function import the 'dolfin-doc' module
>>>> and copy the doc-strings from all classes and functions to the classes
>>>> and functions in the real dolfin module as was suggested by Johan
>>>> Hake.
>>>>
>>>> The problem with this approach is that assigning to __doc__ is not
>>>> permitted for objects of 'type' type.
>>>
>>> :(
>>>
>>> I did not anticipate this. Not sure why this is. I have got the impression
>>> that numpy get around this. They use numpydoc to dynamically add their
>>> documentation. It makes heavy use of sphinx, but I couldn't figure how they
>>> get around that __doc__ is read-only.
>>
>> To me it looks like numpydoc is a Sphinx extension that translates the
>> Numpy docstrings into something that Sphinx can understand, not the
>> other way around which is what we want.
>>
>> http://projects.scipy.org/numpy/browser/trunk/doc/sphinxext/README.txt
>>
>> So I think our best bet is to proceed with your suggestions below.
>>
>> Kristian
>>
>>> While it might be cool to look into what NumPy have done, (they also define a
>>> pseudo classes, which they populate with docstrings, (look into
>>> phantom_import.py), and they also define some nice format for the reST used in
>>> the docstrings), I suggest two things we can do:
>>>
>>> 1) SWIG can generate docstrings. We do that allready using parsed doxygen
>>> documentation. All of this is gathered in docstrings.i. I suggest generating
>>> such a file from our documentation. We need to turn of %feature("autodoc","1")
>>> in dolfin.i to get rid of the long and sometimes faulty generated signatures.
>
> I turns out that it's only the __doc__ of the class I can't assign to,
> not the __doc__ of member functions (and regular functions).
> A simpler solution (at least for me) is to parse the cpp.py module
> once generated and substitute all docstrings of classes with the
> docstrings from the dolfindoc module rather than creating the
> docstrings.i file.
>
> Then for the classes that we manually add we use the method you
> described below, but only for class.__doc__ .
>
> class Foo(object):
>    __doc__ = dolfindoc.Foo.__doc__
>    def bar(self):
>        "this doc string will be substituted with the
> dolfindoc.Foo.__dict__["bar"].__doc__."
>        pass

Unfortunately it turned out I was too quick to make this conclusion.
For instance,
for the Swig generated class Mesh in the module cpp I can't assign to

Mesh.__doc__

because it is a 'type' object as pointed our earlier, I can assign to:

Mesh.__dict__["__init__"].__doc__

but not

Mesh.__dict__["size"].__doc__

along with practically all other member functions.

The reason is that Swig, after the class definitions does this:

Mesh.size = new_instancemethod(_cpp.Mesh_size,None,Mesh)

to which no assignment is possible.
'AttributeError: attribute '__doc__' of 'instancemethod' objects is
not writable'
The code in Mesh.size is already

    def size(self, *args):
        return _cpp.Mesh_size(self, *args)

so why Swig overrides the method I don't know. If it didn't I could
assign to the docstring.

Recall that my initial plan was to simply parse the cpp.py file and
substitute the docstrings of all classes with what the pseudo module
dolfindoc would define. However, that doesn't seem so practical
anymore since I would need to also comment out all lines where
instancemethods are being assigned to class members such that on
import (in dolfin/__init__.py) I can loop classes and assign to
function docstrings. This is of course possible but seems a bit
awkward and I don't know what implications the commenting out of
instancemethod assignment will have.
As I see it, two solutions remain.

1) Take the pseudo module dolfindoc from the FEniCS documentation and
create a docstring.i generator. The script
dolfin/dolfin/swig/generator.py should then try to import the
dolfindoc module, if successful use that to create docstring.i,
otherwise use the Doxy2SWIG generator.
One question w.r.t this approach, when is docstrings.i being
generated? I see it is distributed with DOLFIN but shouldn't it be
generated everytime DOLFIN is rebuild with enablePython, or does
enableDocs have to be switched on as well?

2) Extend our Python layer with all functions and classes that we want
to use the dolfindoc docstrings for.
These should just be empty and redirect calls to classes and functions
of the cpp. In dolfin/mesh/mesh.py:

class Mesh(cpp.Mesh):
    def size(self):
        try:
            import dolfindoc.mesh.mesh
            __doc__ = dolfindoc.mesh.mesh.Mesh.__doc__
        except:
            __doc__ = cpp.Mesh.__doc__
        return cpp.Mesh.size()

An additional benefit of this approach is that the module structure
can be identical to what we have in the _real_ DOLFIN, not as it is
now where everything is dumped in the dolfin.cpp module.
I don't know how much overhead this will create, alternatively we can
skip the try/except clause and simply have the documentation as a
dependency, or not add the docstrings for memberfunctions and add them
later on import as was the original idea.

Suggestions and comments are more than welcome!

Kristian

> then in dolfin/__init__.py we load the classes as we do now from the
> dolfin module, and then iterate over all functions and member
> functions and substitute docstrings from the dolfindoc module.
>
> Kristian
>
>>> 2) The added python classes and methods can be documented using your suggested
>>> approach, but instead of adding the docstring after class creation, do it
>>> during class (method or function) creation, a la:
>>>
>>>  class Foo(object):
>>>      __doc__ = docstrings.Foo.__doc__
>>>      ...
>>>
>>> where docstrings is the generated module containing the docstrings.
>>>
>>> Johan
>>>
>>>> In other words we can't assign to the __doc__ of
>>>>
>>>> class Foo(object):
>>>>     "Foo doc"
>>>>     pass
>>>>
>>>> Which is a new-style class and found in UFL and the SWIG code in DOLFIN.
>>>>
>>>> It works fine for
>>>>
>>>> def some_function(v):
>>>>     "function doc"
>>>>     return 2*v
>>>>
>>>> and
>>>>
>>>> class Bar:
>>>>     "Bar doc"
>>>>     pass
>>>>
>>>> which is the old-style class often found in FFC.
>>>>
>>>> Does anyone have a solution or comments to the above approach, or
>>>> maybe we can do it in a completely different way.
>>>>
>>>> I read about some workaround for the 'assign to __doc__' problem, but
>>>> it doesn't seem that nice and it might be a problem to incorporate in
>>>> the SWIG generated code?
>>>>
>>>> http://stackoverflow.com/questions/71817/using-the-docstring-from-one-metho
>>>> d-to-automatically-overwrite-that-of-another-me
>>>>
>>>>
>>>> Kristian
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~fenics
>>>> Post to     : fenics@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~fenics
>>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>