dolfin team mailing list archive

[Anders Logg <logg@xxxxxxxxx>]

On Thu, Jan 24, 2008 at 10:07:16AM +0100, Martin Sandve Alnæs wrote:
> I've tried to implement multicore assembly using openmp in dolfin, but
> it was a big failure. I've attached the hg bundle. The code is
> protected by #ifdef _OPENMP, so it should be safe to merge into dolfin
> if anyone wants to pursue this further (I won't).

We're currently doing some major surgery on the Assembler, DofMap and
SparsityPattern classes for parallel assembly (using MPI) so it's
probably not a good time to merge it in right now. In particular if
it doesn't work. ;-)

We probably need to discuss some day how the suggested updates we
(mainly Magnus) have done to (p)DofMap and (p)SparsityPattern to see
if they make sense also for your Epetra matrices. We have used PETSc.

-- 
Anders


> To compile with openMP, I did:
> CXX=g++-4.2 CXXFLAGS='-fopenmp -O3' ./configure ....
> (I didn't manage to import pydolfin with this build, it missed some symbol.)
> 
> The problem is that the matrix insertion "A.add(...)" must be in a
> critical section such that only one thread inserts at the same time.
> Since the matrix insertion is a dominating part, this only introduces
> a lot of overhead. Although I didn't expect much for the stiffness
> matrix which I tested with, the result is a surprisingly large
> slowdown when running two threads (although both cores were active).
> 
> To fix this, one might split the matrix in one datastructure for each
> thread, and do "communication" between the matrix structures like with
> the MPI-based program. The difference is that the matrices would be in
> the memory of the same process, thus the communication overhead is
> much smaller.