Where do we start?
I guess one good place to start would be to get input/partitioning
working and you seem to have that working already. We should be able
to read in a mesh, partition it (with ParMetis for now) and construct
the MeshFunctions S_i and F_i.
Once that is in place, we can start hacking on DofMapBuilder::build().
Could you outline what you have in terms of input/partitioning and
then we can start applying those patches.
Parallel mesh parsing, the entire mesh is never represented on a single
processor. It's a two stage process, first the coordinates are loaded
and partitioned with a geometric partitioner. In the second stage each
processor loads the cells corresponding to the assigned coordinates, and
finally the mesh is partitioned with a graph partitioner.
Partitioning is done with the distributed geometric and mesh-to-dual
graph partitioner in parmetis.
How does this work? It seems as if you have already partitioned the
domain geometrically, then there's no need to make a partition based on
topology. Or is the geometric partitioning only a first approximation
and then vertices are exchanged between processors?
The coordinates in the XML-file are initially distributed across the
processors with a simple linear distribution, just to get it of the hard
drive.
The geometric partitioner is used to make sure that all coordinates on a
processor lies close to each other. This simplifies and minimize
communication when parsing cell data.
Since the geometric partitioner creates rather bad partitions (with
respected to edge-cut), the graph partitioner is used to create a good
partitioning, with low edge-cut, thus minimizing
data-dependencies/communication.
