On Fri, Jan 04, 2008 at 06:25:11PM +0000, Garth N. Wells wrote:
Anders Logg wrote:
On Fri, Jan 04, 2008 at 06:12:32PM +0000, Garth N. Wells wrote:
Anders Logg wrote:
On Fri, Jan 04, 2008 at 05:50:53PM +0000, Garth N. Wells wrote:
Anders Logg wrote:
On Fri, Jan 04, 2008 at 10:25:35AM -0600, Matthew Knepley wrote:
On Jan 4, 2008 10:22 AM, Garth N. Wells <gnw20@xxxxxxxxx> wrote:
We have a problem at the moment when using PETSc related to conflicts
between dolfin::MPIManager and dolfin::PETScManger as to who should
initialise and finalise MPI. The difficulty is that we can't control the
order in which MPIManager and PETScManager are destroyed.
Given that we will probably have more 'manager' classes in the future
(e.g. for Trilinos), what is the best approach? Some possibilities are
one 'Manager' class that does the lot, or a SingletonManager which can
control the order in which singleton manager classes are destroyed. Ideas?
When using multiple packages with MPI, you should always have a single MPI
manage class. If MPI is already initialized when PETSc is initialized,
it won't mess
with MPI (and won't finalize it either).
Matt
Good, if it's always the case that PETSc does not finalize when it has
not initialized, then maybe we just need to do the same?
As we have implemented it, MPIManager checks if someone else (maybe
PETSc, maybe someone else) has already initialized MPI and in that
case does not initialize it. Then it should also assume that someone
else will finalize it.
We can just add a bool member initialized_here and then do
if (initialized_here)
MPIManager::finalize();
in the destructor of MPIManager.
Would that help?
Unfortunately not.
The problem is that MPIManager might finalize MPI before PETSc has been
finalised. If MPI is initialised before PETSc, then PETSc should be
finalised before MPI is finalised. The problem is that we have no
control over the order in which MPIManager and PETScManager are destroyed.
ok.
I'm thinking of a singleton class DolfinManager which is responsible for
the creation and destruction of various managers in the appropriate order.
Garth
Perhaps it would be better to have a single class that takes care of
all initializations (rather than having a class that takes care of
manager classes) to keep things simple?
ok.
We could put a class named Init (for example) in src/kernel/main/
with some static functions:
static void init(int argc, char* argv[]);
static void initPETSc();
static void initPETSc(int argc, char* argv[]);
static void initMPI();
We can then remove init.cpp, init.h and also PETScManager.
ok.
MPIManager can be renamed to MPI and just contain MPI utility
functions (like everything in MPIManager now except init/finalize).
What about calling it DolfinManager as it won't be strictly for MPI?
Without MPI, we still need to initialise PETSc.
Garth
The thing I suggest calling MPI is strictly for MPI (the things
currently in MPIManager except init/finalize).
Do you still propose having a class MPI?
Yes, and it contains the following things:
/// Return proccess number
static uint processNumber();
/// Return number of processes
static uint numProcesses();
/// Determine whether we should broadcast (based on current parallel policy)
static bool broadcast();
/// Determine whether we should receive (based on current parallel policy)
static bool receive();
I agree the class that manages initialization should be called
something neutral. I'm not very fond of "DolfinManager" since (1)
maybe it should then be DOLFINManager (which is maybe not very nice)
and (2) it is not something that manages DOLFIN.
Suggestions? SubSystemManager?
Sounds good.
For simplicity, we could also have a class named PETSc with a static
init() function that would just call SubSystemManager. Since we need
to call this in all PETSc data structures, it's convenient if we can
write
PETSc::init()
rather than
SubSystemManager::initPETSc();
Similarly, we can have an init() function in the class MPI that calls
SubSystemManager::initMPI().
So three classes:
SubSystemManager: singleton manager of all subsystem with some
logic for the order of initialization/finalization
MPI: MPI convenience class
PETSc: PETSc convenience class
