dolfin team mailing list archive

[Johan Hake <hake@xxxxxxxxx>]

On Wednesday 17 December 2008 19:20:11 Anders Logg wrote:
> Ola and I have now finished up the first round of getting DOLFIN to
> run in parallel. In short, we can now parse meshes from file in
> parallel and partition meshes in parallel (using ParMETIS).
>
> We reused some good ideas that Niclas Jansson had implemented in
> PXMLMesh before, but have also made some significant changes as
> follows:
>
> 1. The XML reader does not handle any partitioning.
>
> 2. The XML reader just reads in a chunk of the mesh data on each
> processor (in parallel) and stores that into a LocalMeshData object
> (one for each processor). The data is just partitioned in blocks so
> the vertices and cells may be completely unrelated.
>
> 3. The partitioning takes place in MeshPartitioning::partition,
> which gets a LocalMeshData object on each processor. It then calls
> ParMETIS to compute a partition (in parallel) and then redistributes
> the data accordingly. Finally, a mesh is built on each processor using
> the local data.
>
> 4. All direct MPI calls (except one which should be removed) have been
> removed from the code. Instead, we mostly rely on dolfin::MPI::distribute
> which handles most cases of parallel communication and works with STL
> data structures.
>
> 5. There is just one ParMETIS call (no initial geometric
> partitioning). It seemed like an unnecessary step, or are there good
> reasons to perform the partitioning in two steps?
>
> For testing, go to sandbox/passembly, build and then run
>
>   mpirun -n 4 ./demo
>   ./plot_partitions 4

Looks beautiful!

I threw a 3D mesh of 160K vertices onto it, and it was partitioned nicely in 
some 10 s, on my 2 core laptop.

Johan

> Everyone interested in the parallel design and implementation is
> encouraged to have a look at the following new classes:
>
>   LocalMeshData:    storage for local mesh data (not yet a mesh)
>   XMLLocalMeshData: parallel parser for local mesh data
>   MeshPartitioning: partitioning algorithm, calling ParMETIS