← Back to team overview

dolfin team mailing list archive

Re: DofMapSet design

 

Anders Logg wrote:
On Mon, Aug 18, 2008 at 11:05:30AM +0200, Niclas Jansson wrote:
Anders Logg wrote:
I think it looks good.

As far as I understand, you build a global numbering of all mesh
entities (which may be different from the local numbering on each
processor), and then the (global parallel) local-to-global mapping
follows from tabulate_dofs just as usual.

So, the difference is that you build a global numbering of the mesh
entities, and we wanted to build a global numbering of the dofs. The
only advantage I can see with our approach is that it may use less
memory, since we don't need to store an extra numbering scheme for all
mesh entities but this not a big deal.

A few questions:

1. Is the above interpretation correct?


Yes.

Another disadvantage with the global numbering scheme is the mesh
connectivity calculations (mesh.init in MeshRenumber).

Why is this a problem? As far as I understand, there are always two
different numberings of mesh entities, one local (same as we have
now) and one global. The local can be computed as usual and then the
global can be reconstructed from the local + the overlap.

(overlap = how the local pieces fit together)



Iterating over the local + overlap requires some mesh connectivity,
which are costly to generate.

What's your point? Are you arguing against a global numbering scheme
for the mesh entities? I thought this is what you have implemented.


I'm not sure if the global numbering scheme is the best approach. It worked well for simple dof_maps / elements, with a low renumbering time. But for a more general implementation, renumbering starts to take too much time.


6. Does it work for assembly over interior facets (like in DG
methods)?


I'm not sure, haven't tried any DG type problems.

I expect it to be problematic. For interior facet integrals, we need
to iterate over all interior facets in the mesh and for each facet
access its two adjacent cells. One of these may be on another
processor.


But shouldn't a set of ghosted cells for the interior boundary solve
this problem?

Do you mean an overlap consisting of cells, not facets?


Yes, cells. Otherwise, how should a processor access a cell on a adjacent processor?


Then some suggestions:

1. I'd like to move the implementation of DofMap::build() to
DofMapBuilder (to simplify DofMap.cpp).

2. Function names should be fooBar(), not foo_bar().


I recently found doc/misc/policy :)

3. There needs to be some #ifdef HAS_PARMETIS so it may be built
without Parmetis.

4. I'd like to add a new class GlobalNumbering (to replace
MeshDistributedData) that holds the global numbering scheme.

The Mesh class can have a pointer to a GlobalNumbering object which is
0 by default so no extra data (or at least not more than 4 bytes) is
stored when not running in parallel.

Then we can add a function MeshEntity::number() which returns the same
as index() if GlobalNumbering is 0. Otherwise, it returns what is
stored in GlobalNumbering.

GlobalNumbering can have an array of MeshFunctions, one for each
topological dimension, that maps the local entity indices to their
global numbers.

Thus, a MeshEntity will have two functions index() and number().
These will return the same value in sequential and possibly different
values in parallel.

Let's await some more comments and then get started. It would be nice
to get it in small patches to give us an opportunity to comment/edit.
Adding GlobalNumbering and MeshEntity::number() would be a good start.


Sounds good, However I think the {T,S,F,V,O} design mentioned by Johan
is more efficient.

These are different issues. One issue is how to represent the overlap,
however we choose to do that, either {T,S,F,V,O} or just {T, S, F}.
The other issue is how we store either the global mesh numbering or
the global dof map. We need to discuss both.

I'd like to continue this discussion for some time before we jump on
to either option.

1. Do we want to compute a global mesh numbering and then let the
global dof map follow as usual, or do we want to have just a local
mesh numbering and then compute a global dof map from the local dof
maps + the overlap?

In either case, the resulting global dof map can be renumbered to
reduce bandwidth/communication.

2. How do we want to represent the overlap? Is it enough to store it
as {T, S, F} or do we need more? Do we need a class Overlap that
stores the overlap?

I think the {V,O} information is needed in order to redistribute
(reconstruct with the MeshEditor) a mesh during computation (due to load
balancing). A Global numbering is (probably) also needed for
refinement/coarsening.

Again, aren't these separate issues? One issue is how we compute the
global dof map (which is needed for assembly). The other is how we
store information about the overlap which we need both for computing a
global numbering or global dof map, and for load balancing,
refinement, coarsening etc.

Other opinions?
Do we want to (i) build a global numbering of mesh entities or (ii)
have local numbering of mesh entities and build a global dof map?

--
Anders

True, they are different issues, but I think the choice of dof map method should consider how expensive the overlap information are to compute, and if it could be done with an already distributed mesh.

Niclas


Follow ups

References