← Back to team overview

dolfin team mailing list archive

  1. dolfin team
  2. Mailing list archive
  3. Message #09244

Re: DofMapSet design

  Thread Previous • Date Previous • • Thread Next 


Anders Logg wrote:

On Thu, Aug 21, 2008 at 09:10:03AM +0200, Niclas Jansson wrote:

Anders Logg wrote:

On Wed, Aug 20, 2008 at 06:17:30PM +0200, Niclas Jansson wrote:


Stage 2 seems to involve a lot of communication, with small messages.
I think it would be more efficient if the stage were reorganized such
that all messages could be exchanged "at once", in a couple of larger
messages.

That would be nice. I'm very open to suggestions.

If understand the {T, S, F} overlap correctly, a facet could be globally
identified by the value of F(facet).

No, F(facet) would be the local number of the facet in subdomain S(facet).


If so, one suggestion is to buffer N_i and F(facet) in 0...p-1 buffers
(one for each processor) and exchange these during stage 2.

-- stage 1

  for each facet  f \in T
    j = S_i(f)
    if j > i

        -- calculate dof N_i

        buffer[S_i(f)].add(N_i)
        buffer[S_i(f)].add(F_i(f))
    end
  end


-- stage 2

-- Exchange shared dofs with fancy MPI_Allgatherv or a lookalike
-- MPI_SendRecv loop.

   for j = 1 to j = (num processors - 1)
      src = (rank - j + num processors) % num processors
      dest = (rank + j) % num processors

      MPI_SendRecv(dest, buffer[dest], src, recv_buffer)

      for i = 0 to sizeof(recv_buffer), i += 2
         --update facet recv_buff(i+1) with dof value in  recv_buff(i)
      end

   end

I didn't look at this in detail (yet). Is it still valid with the
above interpretation of F(facet)?


Yes, I think so.


I think I understand your point, but I don't understand the details
of your code.

The mapping N_i is an auxiliary global-to-global mapping, which maps
the global dofs on a local mesh to global dofs on the global mesh. It
has a meaning only on each local mesh. What we want to communicate is
the stuff in M_i.

Should we try to implement this? It will essentially be Algorithm 5++
(Algorithm 5 with your improvements). So we don't store a global
numbering of mesh entities but instead compute a global dof map in
parallel. And we store the overlap as MeshData in some way (a set of
MeshFunctions attached to each local mesh). I'm very open to which set
of MeshFunctions we will need, either just S, F or additional data we
might need.

Other opinions? Garth? Ola?
I haven't been following the details closely enough to make much of a comment. This thread is not easy to follow, so I created a wiki page 

  http://www.fenics.org/wiki/DOLFIN_parallel
for listing what's required and the proposed strategy. Let's list what we want (and prioritise if necessary) and how we propose to do it. Then we'll be able to break it into tasks. 

Garth






------------------------------------------------------------------------

_______________________________________________
DOLFIN-dev mailing list
DOLFIN-dev@xxxxxxxxxx
http://www.fenics.org/mailman/listinfo/dolfin-dev

References

  Thread Previous • Date Previous • • Thread Next