dolfin team mailing list archive

[Johan Hake <johan.hake@xxxxxxxxx>]

[snip]

> > Sure, I was about to commit this evening but this somewhat turned me off
> > ;)
> 
> Are you going to push it?

Tonight! Sorry for the delay.

> >>> I see your point of having a manageable SWIG interface. It's a strong
> >>> one. However the lack of compatibility with NumPy still exists.
> >>
> >> Maintainability has to be the overriding consideration.
> >
> > Yes, I also think this is a strong goal, and I have therefore spent a lot
> > of time documenting the interface files.
> 
> The documentation helps me a lot in how to use something, but it hasn't
> helped (and nor could it) when making changes.

Ok, that is good!
 
> >> I have spent a
> >> lot of time trying (and sometimes occasionally succeeding) to make just
> >> small updates and changes. The time required is not sustainable.
> >
> > No I agree that std::vector and home made typemaps was a bit over the
> > top. But the change to std::vector was a decision that was taken in a
> > thread, with no or few considerations for the SWIG interface.
> 
> We actually had SWIG/Python in mind when making the change, the idea at
> the time being that by using std::vector, x would carry information on
> its dimension which was previously inferred by through the Mesh. I'm not
> sure that this is valid still in the light of recent changes.

I could have told you. But I probably have to blame my self, as I did not put 
it strong enough when the issue was up. Difficult to follow the pace of 
DOLFIN, with you and Anders being so trigger happy ;) 

> >> Also,
> >> the difficulty of working with the SWIG interface will make it near
> >> impossible to engage new contributors/maintainers.
> >
> > Sure, I just think it was a bit drastic to just remove it instead of
> > having a discussion on a Blueprint or something.
> 
> Now that I think about Launchpad, I should have used the fancy new
> feature to publish a branch along side dolfin/main.

Maybe more of the development of larger features should go into such branches?

> >>> NumPy is the natural choice when one want to use contiguous arrays back
> >>> and forth to Python. I would say that a strong design goal is to have a
> >>> Python interface that works smoothly with NumPy, which neither
> >>> std::vector nor std_vector.i do.
> >>
> >> Is there a fundamental reason why std::vector and can't NumPy play well
> >> together?
> >
> > Data ownership is not loose enough in std::vector to play well with NumPy
> > arrays. I think this is the main issue. Ola might add more here? I think
> > it would be natural for Ola to weight in on this issue before such a
> > change.
> >
> >> Since vectors use contiguous arrays, sounds to me like there
> >> is scope for adding a numpy.i file to SWIG.
> >
> > There are such a file, however it works with (int n, double* data) and
> > not std::vector.
> >
> >> It would be nice to have the NumPy integration, but the typemaps are too
> >> complicated for us mere mortals.
> >
> > Yes, and FE coding is way too complicated for me to know all the nit-bits
> > in so I am glad there are other people taking care of this.
> 
> Importantly, there is more than one person active on the FE side. We
> need more than one person with the ability to work on the SWIG and
> Python sides. In the absence of able people, we need to make it simple
> enough that a least a few of us can take care of it.

Yes, I see that. I was just grumpy yesterday :)

> If you ever took a
> 
> > look into std_vector.i you would see _a lot_ of un-understandable things
> > too.
> 
> Yep, but I don't have to look, just like I don't read the STL file
> 'vector'. I just use it and it works

:)

> > I think we had a manageable interface before the std::vector change. But
> > that's just my 2 cent.
> 
> For better or worse, we've generally followed a principle of designing
> what we think is the best C++ interface and then let SWIG and Python
> follow. It's worked well so far, with a some C++ constructs added and
> designs modified designs to make the Python side work ultimately being
> unnecessary and therefore removed.

Yes, and maybe we need to think more of how this could be done in SWIG ahead 
of any larger changes? So the two interfaces could follow each other a bit 
better?

> >>> A better solution, in my point, would either be to stay with
> >>> double/uint * for the communication of contiguous arrays, or to add a
> >>> light weight wrapped class in C++ for these data structures.
> >>
> >> From the C++ side, I would like to see plain pointers removed from the
> >> user interface.
> >
> > Yes, and I agree. It is preferable from a Python side of view too. But I
> > would prefer it over a std::vector interface in Python anytime. There
> > might also be other solutions, which we have not tried. Heck, for all I
> > know, going with std_vector.i might be the best overall solution, but I
> > am _really_ not convinced.
> 
> I'm wondering if std::vector and plain pointers in the interface are the
> right thing to be wrapping with NumPy. We're generally just exchanging
> coordinate data and function values in and out. 

I see you point. And it might be enough. I just got frustrated working with 
the SWIG vector object in Python as it is not very Pythonic and it creates a 
lot of code for each type we want to wrap, which in turn makes the compilation 
take longer, together with larger binaries.

Feel free to get it working in the other branch. Before we settle on whether 
we go for this solution or we add a SimpleVector class, as sketched by Anders, 
it would be nice to have opinions from Kent and Ola too.

> Should it instead be
> Vectors and Matrices that interface to NumPy? We can access the raw
> pointers for these objects to create NumPy objects, and we can create
> Vectors and Matrices from pointers, for example using
> 
>   VecCreateSeqWithArray
>   MatCreateSeqAIJWithArrays

I think that this should be the minimum standard and it works quite nicely 
already I think.

> Whatever the case may be, things have been broken for some days now so
> we settle on a fix to get back on track.

Yup on to it this evening :)

Johan