dolfin team mailing list archive

[Anders Logg <logg@xxxxxxxxx>]

On Fri, Jan 04, 2008 at 06:25:11PM +0000, Garth N. Wells wrote:
> 
> 
> Anders Logg wrote:
> > On Fri, Jan 04, 2008 at 06:12:32PM +0000, Garth N. Wells wrote:
> >>
> >> Anders Logg wrote:
> >>> On Fri, Jan 04, 2008 at 05:50:53PM +0000, Garth N. Wells wrote:
> >>>> Anders Logg wrote:
> >>>>> On Fri, Jan 04, 2008 at 10:25:35AM -0600, Matthew Knepley wrote:
> >>>>>> On Jan 4, 2008 10:22 AM, Garth N. Wells <gnw20@xxxxxxxxx> wrote:
> >>>>>>> We have a problem at the moment when using PETSc related to conflicts
> >>>>>>> between dolfin::MPIManager and dolfin::PETScManger as to who should
> >>>>>>> initialise and finalise MPI. The difficulty is that we can't control the
> >>>>>>> order in which MPIManager and PETScManager are destroyed.
> >>>>>>>
> >>>>>>> Given that we will probably have more 'manager' classes in the future
> >>>>>>> (e.g. for Trilinos), what is the best approach? Some possibilities are
> >>>>>>> one 'Manager' class that does the lot, or a SingletonManager which can
> >>>>>>> control the order in which singleton manager classes are destroyed. Ideas?
> >>>>>> When using multiple packages with  MPI, you should always have a single MPI
> >>>>>> manage class. If MPI is already initialized when PETSc is initialized,
> >>>>>> it won't mess
> >>>>>> with MPI (and won't finalize it either).
> >>>>>>
> >>>>>>   Matt
> >>>>> Good, if it's always the case that PETSc does not finalize when it has
> >>>>> not initialized, then maybe we just need to do the same?
> >>>>>
> >>>>> As we have implemented it, MPIManager checks if someone else (maybe
> >>>>> PETSc, maybe someone else) has already initialized MPI and in that
> >>>>> case does not initialize it. Then it should also assume that someone
> >>>>> else will finalize it.
> >>>>>
> >>>>> We can just add a bool member initialized_here and then do
> >>>>>
> >>>>>   if (initialized_here)
> >>>>>     MPIManager::finalize();
> >>>>>
> >>>>> in the destructor of MPIManager.
> >>>>>
> >>>>> Would that help?
> >>>>>
> >>>> Unfortunately not.
> >>>>
> >>>> The problem is that MPIManager might finalize MPI before PETSc has been 
> >>>> finalised. If MPI is initialised before PETSc, then PETSc should be 
> >>>> finalised before MPI is finalised. The problem is that we have no 
> >>>> control over the order in which MPIManager and PETScManager are destroyed.
> >>> ok.
> >>>
> >>>> I'm thinking of a singleton class DolfinManager which is responsible for 
> >>>> the creation and destruction of various managers in the appropriate order.
> >>>>
> >>>> Garth
> >>> Perhaps it would be better to have a single class that takes care of
> >>> all initializations (rather than having a class that takes care of
> >>> manager classes) to keep things simple?
> >>>
> >> ok.
> >>
> >>> We could put a class named Init (for example) in src/kernel/main/
> >>> with some static functions:
> >>>
> >>>   static void init(int argc, char* argv[]);
> >>>
> >>>   static void initPETSc();
> >>>   static void initPETSc(int argc, char* argv[]);
> >>>
> >>>   static void initMPI();
> >>>
> >>> We can then remove init.cpp, init.h and also PETScManager.
> >>>
> >> ok.
> >>
> >>> MPIManager can be renamed to MPI and just contain MPI utility
> >>> functions (like everything in MPIManager now except init/finalize).
> >>>
> >> What about calling it DolfinManager as it won't be strictly for MPI? 
> >> Without MPI, we still need to initialise PETSc.
> >>
> >> Garth
> > 
> > The thing I suggest calling MPI is strictly for MPI (the things
> > currently in MPIManager except init/finalize).
> >
> 
> Do you still propose having a class MPI?

Yes, and it contains the following things:

    /// Return proccess number
    static uint processNumber();

    /// Return number of processes
    static uint numProcesses();

    /// Determine whether we should broadcast (based on current parallel policy)
    static bool broadcast();

    /// Determine whether we should receive (based on current parallel policy)
    static bool receive();

> > I agree the class that manages initialization should be called
> > something neutral. I'm not very fond of "DolfinManager" since (1)
> > maybe it should then be DOLFINManager (which is maybe not very nice)
> > and (2) it is not something that manages DOLFIN.
> > 
> 
> Suggestions? SubSystemManager?

Sounds good.

For simplicity, we could also have a class named PETSc with a static
init() function that would just call SubSystemManager. Since we need
to call this in all PETSc data structures, it's convenient if we can
write

  PETSc::init()

rather than

  SubSystemManager::initPETSc();

Similarly, we can have an init() function in the class MPI that calls
SubSystemManager::initMPI().

So three classes:

   SubSystemManager: singleton manager of all subsystem with some
   logic for the order of initialization/finalization

   MPI: MPI convenience class

   PETSc: PETSc convenience class
  

--
Anders