dolfin team mailing list archive

[Johan Hake <hake@xxxxxxxxx>]

On Wednesday 18 November 2009 18:03:16 Garth N. Wells wrote:
> Johan Hake wrote:
> > On Monday 16 November 2009 22:03:13 DOLFIN wrote:
> >> One or more new changesets pushed to the primary dolfin repository.
> >> A short summary of the last three changesets is included below.
> >>
> >> changeset:   7459:739b7539d6eb
> >> tag:         tip
> >> user:        "Garth N. Wells <gnw20@xxxxxxxxx>"
> >> date:        Mon Nov 16 21:02:54 2009 +0000
> >> files:       dolfin/swig/numpy_typemaps.i
> >> dolfin/swig/std_vector_typemaps.i site-packages/dolfin/expression.py
> >> description:
> >> Get typemap(in) for std::vector working.
> >>
> >> Seems to work but I don't know if it's any good.
> >
> > (Resending the email as I forgot to send it using the correct email
> > adress...)
> >
> > I haven't got the time to look at it, but yes the provided typemaps do
> > make a copy, which is not optimal. I have have actually been a bit
> > worried ;) that you might change the interface of eval, and possible
> > other methods too, from double* to std::vector. In most perspectives it
> > is much nicer to pass a std::vector, not least for safety. However
> > std::vector is more difficult to interact with NumPy.
> 
> double* is easier, but my impression is that it's not particularly safe.

I agree. Getting rid of the occasional segfaults that are tricky to debug, and 
certainly not understandable for pure Python users, is tractable.

> > SWIG do provide it's own implementation of std::vector which more or less
> > wrap the c++ interface of a vector seamless. We have used this to pass
> > arguments of std:vector previously. However the amount of code that is
> > produced and not used is large (~1MB for DOLFIN version < 9.2). NumPy
> > arrays and lists are also much more convenient structures to work with in
> > Python.
> 
> Does the SWIG implementation of std::vector involve copying? I expect
> that it does because I don't see how one can create a
> std::vector<double> from a list since that would require creating a
> std::vector that points to someone else's (the list's) data. It would
> break the std::vector design.

I actually think it does not involve copying. They have reimplemented the 
whole std::vector interface, and integrated it into SWIG type system. That 
goes for many of the containers in the standard library. How the communication 
with the original C++ library is done is beyond me.

> I see how it could work if one creates a DoubleVector rather than a
> list, but that's ugly.

I agree. And I have put quite a lot of effort to remove the dependency of 
std_vector.i. But if we decide that this is a show stopper we can revert to 
std_vector.i. I think we then need to do that for all std::vector typemaps, 
including the pointer typemaps for DirichletBC, GenericFunction, FunctionSpace 
aso.

> > I have therefore decided to remove the dependency of std_vector.i by
> > adding our own typemaps for std::vector arguments. Each one of those
> > typemaps do make a copy. For now we have typmaps for primitives and
> > pointers so each copy should not be that bad. But for operations that is
> > performed alot of times, it might be a difference. But I doubt it would
> > for the callback functionality, as I assume just calling the python
> > function would take most of the time.
> 
> Agree.

Ok.

> > We could consider creating a NumPy array from the pointer to the first
> > element of the vector, as I have seen you have discussed on the list
> > previously, for other cases. We then have to consider the life time of
> > the vector.
> 
> We need a std::vector that stores a shared_ptr to its data ;).

Yeah, that would be nice.

> > For
> > director typemaps that would probably be ok. For other "out" typemaps it
> > might not be ok.
> 
> I would be inclined to make a copy. It's safe and we can document it so
> if one sends a NumPy array into a function that takes a
> std::vector<double>&, and that function stores a reference to the
> object, then it 's not the original NumPy array (confusing???).

Thats ok, for all pure in typemaps, for example the coordinates. However when 
one expects an out argument, like the values in an eval method, we need to 
change any provided values. This could be done in a layer on top of the 
typemap. Then we could make the output NumPy array optional. Like we now have 
in the __call__ method. 

> Do we presently copy data between NumPy arrays and the sub-classes of
> GenericVector?

For the foo.array() method, yes. But for MTL4 and uBLAS backend we provide a 
NumPy array with a pointer to the original data, through the foo.data() 
method. This is consistent with the C++ interface.

> > Would it be an option to add, in C++, a small template wrapper class for
> > arrays that are supposed to go back and forth to Python via NumPy arrays?
> 
> Maybe. I see that this is what Hans Petter does in a chapter of one of
> his books. Still, there are issues with ownership.

Yes, we need to provide some way of passing a view of an other array, using 
such a structure.

> I guess in the end that memory management + pointers is always going to
> be tough in a mixed-language environment.

For sure...

Johan


> 
> Garth
> 
> > Including basic operations so it will be useful also in C++.
> >
> > Johan
> >
> >> changeset:   7458:f9d6aeab7271
> >> user:        "Garth N. Wells <gnw20@xxxxxxxxx>"
> >> date:        Mon Nov 16 19:06:12 2009 +0000
> >> files:       demo/function/eval/python/demo.py
> >>  site-packages/dolfin/expression.py description:
> >> Comment out some dim checks in expression.py.
> >>
> >>
> >> changeset:   7457:6dc505f97225
> >> user:        "Garth N. Wells <gnw20@xxxxxxxxx>"
> >> date:        Mon Nov 16 15:13:08 2009 +0000
> >> files:       demo/function/eval/python/demo.py dolfin/swig/docstrings.i
> >>  dolfin/swig/function_post.i dolfin/swig/function_pre.i
> >>  dolfin/swig/std_vector_typemaps.i description:
> >> Add empty typemap for NumPy array to std::vector<double>.
> >>
> >> ----------------------------------------------------------------------
> >> For more details, visit http://www.fenics.org/hg/dolfin
> >> _______________________________________________
> >> DOLFIN-dev mailing list
> >> DOLFIN-dev@xxxxxxxxxx
> >> http://www.fenics.org/mailman/listinfo/dolfin-dev
> >
> > _______________________________________________
> > DOLFIN-dev mailing list
> > DOLFIN-dev@xxxxxxxxxx
> > http://www.fenics.org/mailman/listinfo/dolfin-dev
>