dolfin team mailing list archive

[Anders Logg <logg@xxxxxxxxx>]

On Fri, Jan 04, 2008 at 06:39:39PM +0000, Garth N. Wells wrote:
> 
> 
> Anders Logg wrote:
> > On Fri, Jan 04, 2008 at 06:25:11PM +0000, Garth N. Wells wrote:
> >>
> >> Anders Logg wrote:
> >>> On Fri, Jan 04, 2008 at 06:12:32PM +0000, Garth N. Wells wrote:
> >>>> Anders Logg wrote:
> >>>>> On Fri, Jan 04, 2008 at 05:50:53PM +0000, Garth N. Wells wrote:
> >>>>>> Anders Logg wrote:
> >>>>>>> On Fri, Jan 04, 2008 at 10:25:35AM -0600, Matthew Knepley wrote:
> >>>>>>>> On Jan 4, 2008 10:22 AM, Garth N. Wells <gnw20@xxxxxxxxx> wrote:
> >>>>>>>>> We have a problem at the moment when using PETSc related to conflicts
> >>>>>>>>> between dolfin::MPIManager and dolfin::PETScManger as to who should
> >>>>>>>>> initialise and finalise MPI. The difficulty is that we can't control the
> >>>>>>>>> order in which MPIManager and PETScManager are destroyed.
> >>>>>>>>>
> >>>>>>>>> Given that we will probably have more 'manager' classes in the future
> >>>>>>>>> (e.g. for Trilinos), what is the best approach? Some possibilities are
> >>>>>>>>> one 'Manager' class that does the lot, or a SingletonManager which can
> >>>>>>>>> control the order in which singleton manager classes are destroyed. Ideas?
> >>>>>>>> When using multiple packages with  MPI, you should always have a single MPI
> >>>>>>>> manage class. If MPI is already initialized when PETSc is initialized,
> >>>>>>>> it won't mess
> >>>>>>>> with MPI (and won't finalize it either).
> >>>>>>>>
> >>>>>>>>   Matt
> >>>>>>> Good, if it's always the case that PETSc does not finalize when it has
> >>>>>>> not initialized, then maybe we just need to do the same?
> >>>>>>>
> >>>>>>> As we have implemented it, MPIManager checks if someone else (maybe
> >>>>>>> PETSc, maybe someone else) has already initialized MPI and in that
> >>>>>>> case does not initialize it. Then it should also assume that someone
> >>>>>>> else will finalize it.
> >>>>>>>
> >>>>>>> We can just add a bool member initialized_here and then do
> >>>>>>>
> >>>>>>>   if (initialized_here)
> >>>>>>>     MPIManager::finalize();
> >>>>>>>
> >>>>>>> in the destructor of MPIManager.
> >>>>>>>
> >>>>>>> Would that help?
> >>>>>>>
> >>>>>> Unfortunately not.
> >>>>>>
> >>>>>> The problem is that MPIManager might finalize MPI before PETSc has been 
> >>>>>> finalised. If MPI is initialised before PETSc, then PETSc should be 
> >>>>>> finalised before MPI is finalised. The problem is that we have no 
> >>>>>> control over the order in which MPIManager and PETScManager are destroyed.
> >>>>> ok.
> >>>>>
> >>>>>> I'm thinking of a singleton class DolfinManager which is responsible for 
> >>>>>> the creation and destruction of various managers in the appropriate order.
> >>>>>>
> >>>>>> Garth
> >>>>> Perhaps it would be better to have a single class that takes care of
> >>>>> all initializations (rather than having a class that takes care of
> >>>>> manager classes) to keep things simple?
> >>>>>
> >>>> ok.
> >>>>
> >>>>> We could put a class named Init (for example) in src/kernel/main/
> >>>>> with some static functions:
> >>>>>
> >>>>>   static void init(int argc, char* argv[]);
> >>>>>
> >>>>>   static void initPETSc();
> >>>>>   static void initPETSc(int argc, char* argv[]);
> >>>>>
> >>>>>   static void initMPI();
> >>>>>
> >>>>> We can then remove init.cpp, init.h and also PETScManager.
> >>>>>
> >>>> ok.
> >>>>
> >>>>> MPIManager can be renamed to MPI and just contain MPI utility
> >>>>> functions (like everything in MPIManager now except init/finalize).
> >>>>>
> >>>> What about calling it DolfinManager as it won't be strictly for MPI? 
> >>>> Without MPI, we still need to initialise PETSc.
> >>>>
> >>>> Garth
> >>> The thing I suggest calling MPI is strictly for MPI (the things
> >>> currently in MPIManager except init/finalize).
> >>>
> >> Do you still propose having a class MPI?
> > 
> > Yes, and it contains the following things:
> > 
> >     /// Return proccess number
> >     static uint processNumber();
> > 
> >     /// Return number of processes
> >     static uint numProcesses();
> > 
> >     /// Determine whether we should broadcast (based on current parallel policy)
> >     static bool broadcast();
> > 
> >     /// Determine whether we should receive (based on current parallel policy)
> >     static bool receive();
> > 
> >>> I agree the class that manages initialization should be called
> >>> something neutral. I'm not very fond of "DolfinManager" since (1)
> >>> maybe it should then be DOLFINManager (which is maybe not very nice)
> >>> and (2) it is not something that manages DOLFIN.
> >>>
> >> Suggestions? SubSystemManager?
> > 
> > Sounds good.
> > 
> > For simplicity, we could also have a class named PETSc with a static
> > init() function that would just call SubSystemManager. Since we need
> > to call this in all PETSc data structures, it's convenient if we can
> > write
> > 
> >   PETSc::init()
> > 
> > rather than
> > 
> >   SubSystemManager::initPETSc();
> > 
> > Similarly, we can have an init() function in the class MPI that calls
> > SubSystemManager::initMPI().
> > 
> > So three classes:
> > 
> >    SubSystemManager: singleton manager of all subsystem with some
> >    logic for the order of initialization/finalization
> > 
> >    MPI: MPI convenience class
> > 
> >    PETSc: PETSc convenience class
> >   
> > 
> 
> OK. I'll add something. We could just let
> 
>    SubSystemManager::init();
> 
> take care of all the initialisation.
> 
> Garth

ok, but shouldn't we be able to initialize one but not the other (of MPI
and PETSc)?

-- 
Anders