dolfin team mailing list archive

[Anders Logg <logg@xxxxxxxxx>]

On Wed, Aug 20, 2008 at 09:39:30AM +0200, Niclas Jansson wrote:
> Anders Logg wrote:
> > On Mon, Aug 18, 2008 at 11:05:30AM +0200, Niclas Jansson wrote:
> >   
> >> Anders Logg wrote:
> >>     
> >>>>> I think it looks good.
> >>>>>
> >>>>> As far as I understand, you build a global numbering of all mesh
> >>>>> entities (which may be different from the local numbering on each
> >>>>> processor), and then the (global parallel) local-to-global mapping
> >>>>> follows from tabulate_dofs just as usual.
> >>>>>
> >>>>> So, the difference is that you build a global numbering of the mesh
> >>>>> entities, and we wanted to build a global numbering of the dofs. The
> >>>>> only advantage I can see with our approach is that it may use less
> >>>>> memory, since we don't need to store an extra numbering scheme for all
> >>>>> mesh entities but this not a big deal.
> >>>>>
> >>>>> A few questions:
> >>>>>
> >>>>> 1. Is the above interpretation correct?
> >>>>>
> >>>>>
> >>>>>           
> >>>> Yes.
> >>>>
> >>>> Another disadvantage with the global numbering scheme is the mesh
> >>>> connectivity calculations (mesh.init in MeshRenumber).
> >>>>
> >>>>         
> >>> Why is this a problem? As far as I understand, there are always two
> >>> different numberings of mesh entities, one local (same as we have
> >>> now) and one global. The local can be computed as usual and then the
> >>> global can be reconstructed from the local + the overlap.
> >>>
> >>> (overlap = how the local pieces fit together)
> >>>
> >>>
> >>>
> >>>       
> >> Iterating over the local + overlap requires some mesh connectivity,
> >> which are costly to generate.
> >>     
> >
> > What's your point? Are you arguing against a global numbering scheme
> > for the mesh entities? I thought this is what you have implemented.
> >
> >   
> 
> I'm not sure if the global numbering scheme is the best approach. It 
> worked well for simple dof_maps / elements, with a low renumbering time. 
> But for a more general implementation, renumbering starts to take too 
> much time.

ok. So do you suggest we implement the other strategy instead, building a
global dof map from local dof maps?

> >>>>> 6. Does it work for assembly over interior facets (like in DG
> >>>>> methods)?
> >>>>>
> >>>>>
> >>>>>           
> >>>> I'm not sure, haven't tried any DG type problems.
> >>>>
> >>>>         
> >>> I expect it to be problematic. For interior facet integrals, we need
> >>> to iterate over all interior facets in the mesh and for each facet
> >>> access its two adjacent cells. One of these may be on another
> >>> processor.
> >>>
> >>>
> >>>       
> >> But shouldn't a set of ghosted cells for the interior boundary solve
> >> this problem?
> >>     
> >
> > Do you mean an overlap consisting of cells, not facets?
> >
> >   
> 
> Yes, cells. Otherwise, how should a processor access a cell on a 
> adjacent processor?

ok, good.

> >>>       
> >>>>> Then some suggestions:
> >>>>>
> >>>>> 1. I'd like to move the implementation of DofMap::build() to
> >>>>> DofMapBuilder (to simplify DofMap.cpp).
> >>>>>
> >>>>> 2. Function names should be fooBar(), not foo_bar().
> >>>>>
> >>>>>
> >>>>>           
> >>>> I recently found doc/misc/policy :)
> >>>>
> >>>>         
> >>>>> 3. There needs to be some #ifdef HAS_PARMETIS so it may be built
> >>>>> without Parmetis.
> >>>>>
> >>>>> 4. I'd like to add a new class GlobalNumbering (to replace
> >>>>> MeshDistributedData) that holds the global numbering scheme.
> >>>>>
> >>>>> The Mesh class can have a pointer to a GlobalNumbering object which is
> >>>>> 0 by default so no extra data (or at least not more than 4 bytes) is
> >>>>> stored when not running in parallel.
> >>>>>
> >>>>> Then we can add a function MeshEntity::number() which returns the same
> >>>>> as index() if GlobalNumbering is 0. Otherwise, it returns what is
> >>>>> stored in GlobalNumbering.
> >>>>>
> >>>>> GlobalNumbering can have an array of MeshFunctions, one for each
> >>>>> topological dimension, that maps the local entity indices to their
> >>>>> global numbers.
> >>>>>
> >>>>> Thus, a MeshEntity will have two functions index() and number().
> >>>>> These will return the same value in sequential and possibly different
> >>>>> values in parallel.
> >>>>>
> >>>>> Let's await some more comments and then get started. It would be nice
> >>>>> to get it in small patches to give us an opportunity to comment/edit.
> >>>>> Adding GlobalNumbering and MeshEntity::number() would be a good start.
> >>>>>
> >>>>>
> >>>>>           
> >>>> Sounds good, However I think the {T,S,F,V,O} design mentioned by Johan
> >>>> is more efficient.
> >>>>
> >>>>         
> >>> These are different issues. One issue is how to represent the overlap,
> >>> however we choose to do that, either {T,S,F,V,O} or just {T, S, F}.
> >>> The other issue is how we store either the global mesh numbering or
> >>> the global dof map. We need to discuss both.
> >>>
> >>> I'd like to continue this discussion for some time before we jump on
> >>> to either option.
> >>>
> >>> 1. Do we want to compute a global mesh numbering and then let the
> >>> global dof map follow as usual, or do we want to have just a local
> >>> mesh numbering and then compute a global dof map from the local dof
> >>> maps + the overlap?
> >>>
> >>> In either case, the resulting global dof map can be renumbered to
> >>> reduce bandwidth/communication.
> >>>
> >>> 2. How do we want to represent the overlap? Is it enough to store it
> >>> as {T, S, F} or do we need more? Do we need a class Overlap that
> >>> stores the overlap?
> >>>
> >>>       
> >> I think the {V,O} information is needed in order to redistribute
> >> (reconstruct with the MeshEditor) a mesh during computation (due to load
> >> balancing). A Global numbering is (probably) also needed for
> >> refinement/coarsening.
> >>     
> >
> > Again, aren't these separate issues? One issue is how we compute the
> > global dof map (which is needed for assembly). The other is how we
> > store information about the overlap which we need both for computing a
> > global numbering or global dof map, and for load balancing,
> > refinement, coarsening etc.
> >
> > Other opinions?
> >   
> > Do we want to (i) build a global numbering of mesh entities or (ii)
> > have local numbering of mesh entities and build a global dof map?
> >
> 
> True, they are different issues, but I think the choice of dof map 
> method should consider how expensive the overlap information are to 
> compute, and if it could be done with an already distributed mesh.

I'm not very particular about how we represent/store the overlap, but
let's solve (1) first.

-- 
Anders

Attachment: signature.asc
Description: Digital signature