dolfin team mailing list archive

[Johan Hake <johan.hake@xxxxxxxxx>]

On Tuesday 01 December 2009 12:20:32 Garth N. Wells wrote:
> Johan Hake wrote:
> > On Tuesday 01 December 2009 12:08:54 Garth N. Wells wrote:
> >> Anders Logg wrote:
> >>> On Tue, Dec 01, 2009 at 07:58:12PM +0000, Garth N. Wells wrote:
> >>>> Anders Logg wrote:
> >>>>> On Tue, Dec 01, 2009 at 07:21:00PM +0000, Garth N. Wells wrote:
> >>>>>> Johan Hake wrote:
> >>>>>>> On Tuesday 01 December 2009 11:06:43 you wrote:
> >>>>>>>> Anders Logg wrote:
> >>>>>>>>> On Tue, Dec 01, 2009 at 09:59:18AM -0800, Johan Hake wrote:
> >>>>>>>>>> On Tuesday 01 December 2009 00:45:50 Anders Logg wrote:
> >>>>>>>>>>> Would it help to add a new class on the C++ side that is used
> >>>>>>>>>>> only for passing array data back and forth between C++ and
> >>>>>>>>>>> Python? We have had this before (SimpleArray) and it would be
> >>>>>>>>>>> fairly easy to extend the C++ with extra functions in the
> >>>>>>>>>>> interface that use SimpleArray instead of std::vector.
> >>>>>>>>>>>
> >>>>>>>>>>> Then perhaps we can have one single typemap that hits
> >>>>>>>>>>> SimpleArray everywhere and converts it to a NumPy array.
> >>>>>>>>>>
> >>>>>>>>>> Yes, something in that direction is what I had in mind. In
> >>>>>>>>>> addition we could also add a foo.array() function to get a NumPy
> >>>>>>>>>> view from this class. This would be nice when we do not want to
> >>>>>>>>>> have all the communication through typemaps, but actually using
> >>>>>>>>>> the SimpleArray in Python as return argument from some function
> >>>>>>>>>> that wants to resize the array.
> >>>>>>>>>>
> >>>>>>>>>> We would also need some stuff to handle memory management.
> >>>>>>>>>>
> >>>>>>>>>> I see two fundamental ways such a class could be used:
> >>>>>>>>>>   1) A replacement for the previous use of double/uint/int*, now
> >>>>>>>>>> std::vector 2) A replacement for communication using std::vector
> >>>>>>>>>> where resize flexibility is needed.
> >>>>>>>>>>
> >>>>>>>>>> I think 1, speaks for it self. 2 is where we need to resize any
> >>>>>>>>>> passed vector. This goes for GenericMatrix.getrow,
> >>>>>>>>>> foo.intersection, GenericFunction.comput_vertex_values.
> >>>>>>>>>>
> >>>>>>>>>>> And the work would be to add the extra stuff on the C++ side.
> >>>>>>>>>>> The advantage would be less complex wrapper code and that Garth
> >>>>>>>>>>> and I are capable of handling the complexities on the C++ side.
> >>>>>>>>>>
> >>>>>>>>>> Yes this must be a goal. I agree that the present SWIG situation
> >>>>>>>>>> has grown out of hands.
> >>>>>>>>>>
> >>>>>>>>>>> But what I don't understand is why it would be easier to write
> >>>>>>>>>>> a typemap for SimpleArray than for std::vector. Both of them
> >>>>>>>>>>> use contiguous memory.
> >>>>>>>>>>
> >>>>>>>>>> Yes, but in std::vector it is now way, I suppose, to prevent a
> >>>>>>>>>> vector to delete its data when it goes out of scope. This is
> >>>>>>>>>> necessary in a typical in typemap.
> >>>>>>>>>
> >>>>>>>>> ok, let's create a very flexible array class that is targeted at
> >>>>>>>>> simple communication between C++ and Python/NumPy. We had a class
> >>>>>>>>> before named SimpleArray. We might call it NumPyArray or
> >>>>>>>>> PythonArray.
> >>>>>>>>
> >>>>>>>> Can we just call it Array? It will be visible in the C++ interface
> >>>>>>>> (e.g. in eval) so it would be good to have a nice name.
> >>>>>>>
> >>>>>>> Agree.
> >>>>>>>
> >>>>>>>>> I have created a blueprint:
> >>>>>>>>>
> >>>>>>>>> https://blueprints.launchpad.net/dolfin/+spec/array-typemaps
> >>>>>>>>
> >>>>>>>> I'll add something. I was thinking already about this. With a
> >>>>>>>> smart pointer to the underlying data we should be to devise an
> >>>>>>>> elegant memory model and be able to tell an Array object when it
> >>>>>>>> does and doesn't own the data upon construction, and be able to
> >>>>>>>> change during execution.
> >>>>>>>
> >>>>>>> Sounds fancy.
> >>>>>>>
> >>>>>>>>> We can fill out the details together.
> >>>>>>>>>
> >>>>>>>>>> I will fix the interface of getrow this evening. I was about to
> >>>>>>>>>> do it yesterday, but instead I got grumpy :) But a good night
> >>>>>>>>>> sleep makes wonders!
> >>>>>>>>>
> >>>>>>>>> Good! :-)
> >>>>>>>>>
> >>>>>>>>> Will you make a fast/temporary fix so that we can get ready for a
> >>>>>>>>> release of 0.9.5 and then we can move the PythonArray
> >>>>>>>>> implementation to a future release?
> >>>>>>>>
> >>>>>>>> The fast fix would be revert back to the
> >>>>>>>>
> >>>>>>>>     eval(double*, std::vector<double>&)
> >>>>>>>>
> >>>>>>>> interface. No point wasting time on typemaps for std::vector if
> >>>>>>>> we're not going to use them.
> >>>>>>>
> >>>>>>> Fell free to make this change, I have a fool proof plan for the
> >>>>>>> other temporary fix though. No new typemaps, just reusing old ones.
> >>>>>>
> >>>>>> OK, we need to make a decision.
> >>>>>>
> >>>>>> Option 1: Revert changes to eval.
> >>>>>>
> >>>>>> Option 2: Get SWIG working with the eval(std::vector<double>&, const
> >>>>>> std::vector<double>&) interface.
> >>>>>>
> >>>>>> What's it going to be? I'm inclined to Option 1 since it requires no
> >>>>>> work and it's been well tested.
> >>>>>
> >>>>> I'm warming up to the Array class option in which case we don't need
> >>>>> to spend more time on fixing typemaps which will be removed anyway.
> >>>>>
> >>>>> So should the plan then be to use Array as much as possible in the
> >>>>> C++ interface, instead of std::vector? We will still use std::vector
> >>>>> and other STL types in the implementation but less of it in the user
> >>>>> interface.
> >>>>
> >>>> Perhaps, as long as we don't end up re-implementing all the
> >>>> std::vector functionality in Array.
> >>>>
> >>>> Garth
> >>>
> >>> I assume we will make it fixed-size which will simplify the
> >>> implementation a great deal and differentiate it from std::vector
> >>> which has push_back.
> >>
> >> Fixed on the C++ side. It should have a function to indicate a change in
> >> size. That way, we can have this process
> >>
> >> 1. An Array is received and turned into a NumPy
> >> 2. Length of NumPy array changes on the Python side
> >> 3. Array is 'informed' of the new length when sent back to C++
> >>
> >> This change in length on the Python side is something we can't handle at
> >> the moment as far as I understand.
> >
> > A NumPy array is already fixed. Any changes wrt length will return a new
> > NumPy array. You can hand slices, which are views of other arrays, but
> > this is different case which we cannot support.
> >
> > If we have a re-sizable Array we can add some flags, so it cannot be
> > resized if it is a view of a passed NumPy array.
> 
> When will a NumPy array destroy its data?

If it owes it data it will be destroyed the moment the number of references to 
the NumPy array in Python reduces to zero. No shared_ptr stuff here.

Johan

> > But shouldn't this discussion be on the Blueprint page?
> 
> Yes.
> 
> Garth
> 
> > Johan
> >
> >> Garth
> >>
> >>> ok, so let's wait for you to revert the changes to eval() and then see
> >>> where we stand.
> >>>
> >>> --
> >>> Anders
> >>
> >> _______________________________________________
> >> Mailing list: https://launchpad.net/~dolfin
> >> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~dolfin
> >> More help   : https://help.launchpad.net/ListHelp
>