dolfin team mailing list archive

[Johan Hake <johan.hake@xxxxxxxxx>]

On Friday February 3 2012 14:14:03 Marie E. Rognes wrote:
> On 02/03/2012 10:07 AM, Johan Hake wrote:
> > On Thursday February 2 2012 18:16:20 Garth N. Wells wrote:
> >> On 2 February 2012 13:09, Marie E. Rognes<meg@xxxxxxxxx>  wrote:
> >>> Thanks for starting to take care of this Johan!
> >>> 
> >>> However, it does not completely resolve the issue for me:
> >>> 
> >>> Building the most recent 1.0.x and trunk on my laptop,
> >>> by clearing the build directory, and running cmake.local
> >>> gave the following (rough) timings for the Python stage:
> >>> 
> >>> 1.0.x (aka 'old'): ~3 min 21 s
> >>> trunk (aka 'new'): ~ 3 min 23 s
> >>> 
> >>> These timings are for not doing anything else at the same
> >>> time except staring at the screen.
> >> 
> >> Did you use 'make -jX'? I would expect building with one process to
> >> take about the same time, but I see is that I can now build the SWIG
> >> interface files in parallel. I haven't timed the difference.
> 
> Yes: cmake.local uses make -j3
> 
> > I use same configurations as in cmake.local (make -j3). My timings are:
> >   1.0.x (aka 'old'): ~1 min 40 s
> >   trunk (aka 'new'): ~ 1 min 55 s
> > 
> > And max resources are as Garth reports.
> 
> Ok, but it still stalls everything else I'm trying to do at the same time.
> 
> > So no, it is not that of an improvement, but I think we have a framework
> > to continue working on. Next step would be to split the modules even
> > further. But we probably have to end up with splitting the directories
> > of la and mesh  to accomplish that.
> 
> One big improvement is that only the modules
> that need to be rebuilt will be each time, right? This is a more
> frequent scenario than building dolfin from scratch, so that should
> save some.

In theory yes. But let see how it works out in practice. Most modules need to 
be aware of types from other modules. Instead of finding what type in that 
module it depends on, I just make each module depend on all the other header 
files, which means...

Johan

> --
> Marie
> 
> > Johan
> > 
> >>> The 'new' still uses so much memory that doing
> >>> anything else at the same time is very painful.
> >> 
> >> I see less<  1GB for each module, whereas before I think that it was
> >> more like>  2GB.
> >> 
> >> Garth
> >> 
> >>> --
> >>> Marie
> >>> 
> >>> On 02/01/2012 05:23 PM, Johan Hake wrote:
> >>>> Hello!
> >>>> 
> >>>> I have now splitted the SWIG interface into 6 combined modules.
> >>>> 
> >>>>    common, io, mesh, function, fem, la
> >>>> 
> >>>> The largest .cxx file generated now is 2MB (for la with all linear
> >>>> algebra backends installed), instead of 6MB. That is still pretty big
> >>>> and take some
> >>>> 1GB of memory during compilation. But the interface can now be
> >>>> generated in
> >>>> several threads, reducing compile time considerable. The runtime
> >>>> memory print,
> >>>> read from a simple top, is not much more than before (some 100MB),
> >>>> which is
> >>>> good.
> >>>> 
> >>>> They new compiled modules all resides under:
> >>>>    dolfin.cpp.foo
> >>>> 
> >>>> All types and functions are now automatically included into:
> >>>>    dolfin.cpp
> >>>> 
> >>>> which means that most modules under site-packages/dolfin still works
> >>>> fine. There are now more types included from the compiled interface by
> >>>> default. If
> >>>> this is a problem we can change that. But now everything is automated.
> >>>> No handpicking done.
> >>>> 
> >>>> The logic in the new SWIG interface is described in:
> >>>>    dolfin/swig/README
> >>>> 
> >>>> One thing to be aware of is that any reference to the compiled
> >>>> interface from
> >>>> another combined module have to be preluded with the name of the
> >>>> module. Example, in la/post.i we call dolfin_error, which is defined
> >>>> in common. Then
> >>>> 
> >>>> we do:
> >>>>    common.dolfin_error(...)
> >>>> 
> >>>> instead of previous:
> >>>>    dolfin_error(...)
> >>>> 
> >>>> Have fun and let the bug reports come raining!
> >>>> 
> >>>> Johan
> >>>> 
> >>>> _______________________________________________
> >>>> Mailing list: https://launchpad.net/~dolfin
> >>>> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
> >>>> Unsubscribe : https://launchpad.net/~dolfin
> >>>> More help   : https://help.launchpad.net/ListHelp
> >>> 
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~dolfin
> >>> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~dolfin
> >>> More help   : https://help.launchpad.net/ListHelp
> >> 
> >> _______________________________________________
> >> Mailing list: https://launchpad.net/~dolfin
> >> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~dolfin
> >> More help   : https://help.launchpad.net/ListHelp
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~dolfin
> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dolfin
> More help   : https://help.launchpad.net/ListHelp