ffc team mailing list archive

["Garth N. Wells" <gnw20@xxxxxxxxx>]

Johannes Ring wrote:
> On Tue, March 31, 2009 10:23, Garth N. Wells wrote:
> > Johan Hake wrote:
> > > On Tuesday 31 March 2009 09:45:33 Johannes Ring wrote:
> > > > On Mon, March 30, 2009 20:19, Johan Hake wrote:
> > > > > On Monday 30 March 2009 20:06:04 Anders Logg wrote:
> > > > > > The buildbot is not responding at the moment so I can't check the
> > > > > > status.
> > > > > All is green except:
> > > > >
> > > > > The DOLFIN macbot how is complaining about the trilinos demo and the
> > > > > DOLFIN
> > > > > linux64-exp which complains about:
> > > > >
> > > > > Traceback (most recent call last):
> > > > >   File "./demo.py", line 8, in <module>
> > > > >     from dolfin import *
> > > > >
> > > > > File
> > > > > "/work/jhbuildbot/fenics/lib/python2.5/site-packages/dolfin/__init__.py",
> > > > > line 16, in <module>
> > > > >     from assemble import *
> > > > >
> > > > > File
> > > > > "/work/jhbuildbot/fenics/lib/python2.5/site-packages/dolfin/assemble.py",
> > > > > line 25, in <module>
> > > > >     import cpp
> > > > >   File
> > > > > "/work/jhbuildbot/fenics/lib/python2.5/site-packages/dolfin/cpp.py",
> > > > > line 25, in <module>
> > > > >     import _cpp
> > > > > ImportError:
> > > > > /work/jhbuildbot/local/lib/openmpi/mca_paffinity_linux.so:
> > > > > undefined symbol: mca_base_param_reg_int
> > > > >
> > > > > I know Johannes has tried to get into this but as far as I know with
> > > > > no
> > > > > success.
> > > > This is a problem with PyDOLFIN and Open MPI 1.3. We (me and Johan) had
> > > > success with a small hack this morning. By adding
> > > >
> > > >   import ctypes
> > > >   ctypes.CDLL('libmpi.so', ctypes.RTLD_GLOBAL)
> > > >
> > > > before loading the cpp module in site-packages/dolfin/__init__.py the
> > > > Python poisson demo ran just fine. Alternatively one should be able to
> > > > use
> > > > this instead:
> > > >
> > > >   import dl
> > > >   import sys
> > > >   flags = sys.getdlopenflags()
> > > >   sys.setdlopenflags(flags | dl.RTLD_GLOBAL)
> > > >
> > > > but because of a bug in the python2.5 package in Hardy (missing dl
> > > > module)
> > > > we couldn't test this.
> > > >
> > > > It might also be possible to do this in C++ directly. Any suggestion on
> > > > the best way to fix this issue? See also:
> > > >
> > > > http://fenics.org/pipermail/deb-dev/2009-March/000210.html
> > > > http://www.open-mpi.org/faq/?category=running#loading-libmpi-dynamically
> > > >
> > > > Open MPI 1.3 is in Debian unstable so it would be great if we could fix
> > > > this before the release.
> > > The point is that the shared mpi library needs to be loaded using the
> > > flag
> > > RTLD_GLOBAL. Otherwise the loaded mpi libraries will be loaded into
> > > private
> > > namespaces causing the undefined symbol link error.
> > >
> > > I do not understand why this is not a problem when only using DOLFIN and
> > > not
> > > PyDOLFIN. Probably the mpi library is only loaded one time to the right
> > > namespace? Well that's how deep my understanding is of this :P
> > Things went sour when DOLFIN was compiled with Trilinos and Trilinos had
> > MPI enabled. Could this be related?
>
> Probably not. Trilinos is built with --disable-mpi on all the buildslaves.

I mean that if this issue is resolved, using --enable-mpi for Trilinos 
will work.

Garth

> Johannes
>
> > Garth
> >
> > > The hack consist of:
> > > Before we load the compiled c++ module, i.e., the _cpp.do, we load the
> > > shared
> > > mpi library using the above mentioned calls. The library stays in memory
> > > and
> > > is not loaded a second time when the _cpp.so module is loaded.
> > >
> > > This is my naive view of the problem and the why the hack works. It
> > > could
> > > probably be solved in a more gracious manner, e.g. before
> > > PetscInitialize is
> > > called in DOLFIN or maybe even better on the PETSc side?
> > >
> > > Cheers!
> > >
> > > Johan
> > > _______________________________________________
> > > FFC-dev mailing list
> > > FFC-dev@xxxxxxxxxx
> > > http://www.fenics.org/mailman/listinfo/ffc-dev