dorsal team mailing list archive

[Johannes Ring <johannr@xxxxxxxxx>]

On Wed, Sep 1, 2010 at 1:52 PM, Anders Logg <logg@xxxxxxxxx> wrote:
> On Wed, Sep 01, 2010 at 01:42:14PM +0200, Johannes Ring wrote:
>> On Wed, Sep 1, 2010 at 1:30 PM, Anders Logg <logg@xxxxxxxxx> wrote:
>> > On Wed, Sep 01, 2010 at 12:22:27PM +0100, Garth N. Wells wrote:
>> >>
>> >>
>> >> On 01/09/10 12:17, Anders Logg wrote:
>> >> > On Wed, Sep 01, 2010 at 01:13:30PM +0200, Anders Logg wrote:
>> >> >> On Wed, Sep 01, 2010 at 12:58:36PM +0200, Johannes Ring wrote:
>> >> >>> On Wed, Sep 1, 2010 at 12:50 PM, Anders Logg <logg@xxxxxxxxx> wrote:
>> >> >>>> On Wed, Sep 01, 2010 at 12:43:12PM +0200, Anders Logg wrote:
>> >> >>>>> On Wed, Sep 01, 2010 at 12:33:40PM +0200, Johannes Ring wrote:
>> >> >>>>>> On Wed, Sep 1, 2010 at 12:12 PM, Anders Logg <logg@xxxxxxxxx> wrote:
>> >> >>>>>>> On Wed, Sep 01, 2010 at 12:09:20PM +0200, Johannes Ring wrote:
>> >> >>>>>>>> On Wed, Sep 1, 2010 at 12:03 PM, Anders Logg <logg@xxxxxxxxx> wrote:
>> >> >>>>>>>>> On Wed, Sep 01, 2010 at 11:36:11AM +0200, Johannes Ring wrote:
>> >> >>>>>>>>>> On Wed, Sep 1, 2010 at 11:13 AM, Harish Narayanan
>> >> >>>>>>>>>> <harish.mlists@xxxxxxxxx> wrote:
>> >> >>>>>>>>>>> On 9/1/10 2:35 PM, Harish Narayanan wrote:
>> >> >>>>>>>>>>>> On 9/1/10 2:09 PM, Johannes Ring wrote:
>> >> >>>>>>>>>>>>> On Wed, Sep 1, 2010 at 9:17 AM, Johannes Ring <johannr@xxxxxxxxx> wrote:
>> >> >>>>>>>>>>>>>> On Wed, Sep 1, 2010 at 8:39 AM, Harish Narayanan
>> >> >>>>>>>>>>>>>> <harish.mlists@xxxxxxxxx> wrote:
>> >> >>>>>>>>>>>>>>> On 9/1/10 12:02 PM, Harish Narayanan wrote:
>> >> >>>>>>>>>>>>>>>> On 9/1/10 2:44 AM, Marie Rognes wrote:
>> >> >>>>>>>>>>>>>>>>> On 31. aug. 2010 22:56, Anders Logg wrote:
>> >> >>>>>>>>>>>>>>>>>> On Tue, Aug 31, 2010 at 04:28:32PM +0200, Marie Rognes wrote:
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>> On 31. aug. 2010 16:22, Johannes Ring wrote:
>> >> >>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>> On Tue, Aug 31, 2010 at 4:13 PM, Marie Rognes <meg@xxxxxxxxx> wrote:
>> >> >>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> I having some issues with PETSc (surprise) as of today.
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> Dorsal tells me that (full log attached):
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> [\snip]
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> -- The following optional packages were found:
>> >> >>>>>>>>>>>>>>>>>>>>> -- (OK) MPI
>> >> >>>>>>>>>>>>>>>>>>>>> -- (OK) PETSC
>> >> >>>>>>>>>>>>>>>>>>>>> -- (OK) SLEPC
>> >> >>>>>>>>>>>>>>>>>>>>> -- (OK) UMFPACK
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> [\snap]
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> But:
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>>>> from dolfin import *
>> >> >>>>>>>>>>>>>>>>>>>>>>>> has_la_backend("PETSc")
>> >> >>>>>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> False
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>>>> A = PETScMatrix()
>> >> >>>>>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>> >> >>>>>>>>>>>>>>>>>>>>>   File "<stdin>", line 1, in <module>
>> >> >>>>>>>>>>>>>>>>>>>>> NameError: name 'PETScMatrix' is not defined
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>> Help?
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>> There is no -DHAS_PETSC in your log. Try to clean up/remove the
>> >> >>>>>>>>>>>>>>>>>>>> dorsal_build_dir inside the DOLFIN source directory.
>> >> >>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>> Removed. Check.
>> >> >>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>> Then rerun Dorsal
>> >> >>>>>>>>>>>>>>>>>>>> and see if the PETSc test passes.
>> >> >>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>> -- The following optional packages were found:
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) MPI
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) PETSC
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) SLEPC
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) UMFPACK
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) PARMETIS
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) GMP
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) CGAL
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) ZLIB
>> >> >>>>>>>>>>>>>>>>>>> -- (OK) PYTHON
>> >> >>>>>>>>>>>>>>>>>>> -- The following optional packages could not be found:
>> >> >>>>>>>>>>>>>>>>>>> -- (**) TRILINOS
>> >> >>>>>>>>>>>>>>>>>>> -- (**) MTL4
>> >> >>>>>>>>>>>>>>>>>>> -- (**) CHOLMOD
>> >> >>>>>>>>>>>>>>>>>>> -- (**) SCOTCH
>> >> >>>>>>>>>>>>>>>>>>> -- Configuring done
>> >> >>>>>>>>>>>>>>>>>>> -- Generating done
>> >> >>>>>>>>>>>>>>>>>>> -- Build files have been written to: /home/meg/local/dorsal_downloads/dolfin
>> >> >>>>>>>>>>>>>>>>>>> make: *** No rule to make target `install'.  Stop.
>> >> >>>>>>>>>>>>>>>>>>> Failure with exit status: 2
>> >> >>>>>>>>>>>>>>>>>>> Exit message: There was a problem building dolfin.
>> >> >>>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> I had the same problem (I think). It turned out to be a problem with my
>> >> >>>>>>>>>>>>>>>>>> PETSC_ARCH variable which was set to linux-gnu-c-opt instead of
>> >> >>>>>>>>>>>>>>>>>> linux-gnu-cxx-opt.
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> Doesn't solve the problem for me :(
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> - My PETSC_ARCH is set to linux-gnu-cxx-opt (cf gentoo.platform)
>> >> >>>>>>>>>>>>>>>>> - cmake reports: -- PETSC_ARCH is linux-gnu-cxx-opt
>> >> >>>>>>>>>>>>>>>>> - Still get same problem.
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> I think this changed in Dorsal at some point recently.
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> Everything updated as of 20 seconds ago ...
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> I too am now seeing strange things such as this:
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> .
>> >> >>>>>>>>>>>>>>>> .
>> >> >>>>>>>>>>>>>>>> .
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> -- PETSC_DIR is /Users/harish/Work/FEniCS/dev/src/petsc-3.1-p4
>> >> >>>>>>>>>>>>>>>> -- PETSC_ARCH is darwin10.0.0-cxx-opt
>> >> >>>>>>>>>>>>>>>> -- Found petscconf.h
>> >> >>>>>>>>>>>>>>>> -- PETSc test failed
>> >> >>>>>>>>>>>>>>>> -- PETSc could not be found. Be sure to set PETSC_DIR and PETSC_ARCH.
>> >> >>>>>>>>>>>>>>>> (missing:  PETSC_TEST_RUNS)
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> .
>> >> >>>>>>>>>>>>>>>> .
>> >> >>>>>>>>>>>>>>>> .
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> -- (OK) MPI
>> >> >>>>>>>>>>>>>>>> -- (OK) MTL4
>> >> >>>>>>>>>>>>>>>> -- (OK) UMFPACK
>> >> >>>>>>>>>>>>>>>> -- (OK) CHOLMOD
>> >> >>>>>>>>>>>>>>>> -- (OK) SCOTCH
>> >> >>>>>>>>>>>>>>>> -- (OK) PARMETIS
>> >> >>>>>>>>>>>>>>>> -- (OK) GMP
>> >> >>>>>>>>>>>>>>>> -- (OK) CGAL
>> >> >>>>>>>>>>>>>>>> -- (OK) ZLIB
>> >> >>>>>>>>>>>>>>>> -- (OK) PYTHON
>> >> >>>>>>>>>>>>>>>> -- The following optional packages could not be found:
>> >> >>>>>>>>>>>>>>>> -- (**) PETSC
>> >> >>>>>>>>>>>>>>>> -- (**) SLEPC
>> >> >>>>>>>>>>>>>>>> -- (**) TRILINOS
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Following which (and independently), CMake fails catastrophically:
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Building CXX object dolfin/CMakeFiles/dolfin.dir/ale/HarmonicSmoothing.cpp.o
>> >> >>>>>>>>>>>>>>> [  1%] cd /foo/dolfin/dorsal_build_dir/dolfin && /opt/local/bin/c++
>> >> >>>>>>>>>>>>>>> -Ddolfin_EXPORTS -DPACKAGE_VERSION=\"0.9.8\" -DBOOST_UBLAS_NDEBUG
>> >> >>>>>>>>>>>>>>> -DHAS_MTL4 -DMTL_HAS_BLAS -DHAS_UMFPACK -DHAS_CHOLMOD -DHAS_SCOTCH
>> >> >>>>>>>>>>>>>>> -DHAS_PARMETIS -DHAS_CGAL -DHAS_GMP -DHAS_ZLIB -DHAS_CPPUNIT -DHAS_MPI
>> >> >>>>>>>>>>>>>>> -DMPICH_IGNORE_CXX_SEEK -frounding-math;-D_REENTRANT -O2 -g -fPIC
>> >> >>>>>>>>>>>>>>> -I/foo/dolfin -I/opt/local/include/libxml2
>> >> >>>>>>>>>>>>>>> -I/Users/harish/Work/FEniCS/dev/include -I/opt/local/include
>> >> >>>>>>>>>>>>>>> -I/Users/harish/Work/FEniCS/dev/src/mtl4 -I/opt/local/include/ufsparse
>> >> >>>>>>>>>>>>>>> -I/opt/local/include/cppunit   -o
>> >> >>>>>>>>>>>>>>> CMakeFiles/dolfin.dir/ale/HarmonicSmoothing.cpp.o -c
>> >> >>>>>>>>>>>>>>> /foo/dolfin/dolfin/ale/HarmonicSmoothing.cpp
>> >> >>>>>>>>>>>>>>> i686-apple-darwin10-g++-4.2.1: no input files
>> >> >>>>>>>>>>>>>>> i686-apple-darwin10-g++-4.2.1: no input files
>> >> >>>>>>>>>>>>>>> Building CXX object
>> >> >>>>>>>>>>>>>>> dolfin/CMakeFiles/dolfin.dir/ale/TransfiniteInterpolation.cpp.o
>> >> >>>>>>>>>>>>>>> cd /foo/dolfin/dorsal_build_dir/dolfin && /opt/local/bin/c++
>> >> >>>>>>>>>>>>>>> -Ddolfin_EXPORTS -DPACKAGE_VERSION=\"0.9.8\" -DBOOST_UBLAS_NDEBUG
>> >> >>>>>>>>>>>>>>> -DHAS_MTL4 -DMTL_HAS_BLAS -DHAS_UMFPACK -DHAS_CHOLMOD -DHAS_SCOTCH
>> >> >>>>>>>>>>>>>>> -DHAS_PARMETIS -DHAS_CGAL -DHAS_GMP -DHAS_ZLIB -DHAS_CPPUNIT -DHAS_MPI
>> >> >>>>>>>>>>>>>>> -DMPICH_IGNORE_CXX_SEEK -frounding-math;-D_REENTRANT -O2 -g -fPIC
>> >> >>>>>>>>>>>>>>> -I/foo/dolfin -I/opt/local/include/libxml2
>> >> >>>>>>>>>>>>>>> -I/Users/harish/Work/FEniCS/dev/include -I/opt/local/include
>> >> >>>>>>>>>>>>>>> -I/Users/harish/Work/FEniCS/dev/src/mtl4 -I/opt/local/include/ufsparse
>> >> >>>>>>>>>>>>>>> -I/opt/local/include/cppunit   -o
>> >> >>>>>>>>>>>>>>> CMakeFiles/dolfin.dir/ale/TransfiniteInterpolation.cpp.o -c
>> >> >>>>>>>>>>>>>>> /foo/dolfin/dolfin/ale/TransfiniteInterpolation.cpp
>> >> >>>>>>>>>>>>>>> Building CXX object dolfin/CMakeFiles/dolfin.dir/common/real.cpp.o
>> >> >>>>>>>>>>>>>>> /bin/sh: -D_REENTRANT: command not found
>> >> >>>>>>>>>>>>>>> /bin/sh: -D_REENTRANT: command not found
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> I made a fix for this yesterday. Maybe you just need to clean up the
>> >> >>>>>>>>>>>>>> CMake cache (remove the dorsal_build_dir folder)?
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> My fix was reverted but it should be fixed again now.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> This problem is fixed, leaving only the petsc/slepc error.
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> I spoke too soon. I now receive the following during the linking stage
>> >> >>>>>>>>>>> (too).
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Undefined symbols:
>> >> >>>>>>>>>>>  "_METIS_mCPartGraphRecursive2", referenced from:
>> >> >>>>>>>>>>>      _ParMETIS_V3_PartKway in libparmetis.a(kmetis.o)
>> >> >>>>>>>>>>>      _PartitionSmallGraph__ in libparmetis.a(weird.o)
>> >> >>>>>>>>>>>      _Mc_InitPartition_RB__ in libparmetis.a(initpart.o)
>> >> >>>>>>>>>>>      _Mc_InitPartition_RB__ in libparmetis.a(initpart.o)
>> >> >>>>>>>>>>>      _Balance_Partition__ in libparmetis.a(initbalance.o)
>> >> >>>>>>>>>>>      _Balance_Partition__ in libparmetis.a(initbalance.o)
>> >> >>>>>>>>>>> ld: symbol(s) not found
>> >> >>>>>>>>>>> collect2: ld returned 1 exit status
>> >> >>>>>>>>>>> make[2]: *** [dolfin/libdolfin.0.9.8.dylib] Error 1
>> >> >>>>>>>>>>> make[1]: *** [dolfin/CMakeFiles/dolfin.dir/all] Error 2
>> >> >>>>>>>>>>> make: *** [all] Error 2
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> I also got this on the Mac buildbot. The problem is that there is a
>> >> >>>>>>>>>> file /opt/local/lib/libmetis.dylib that is used instead of the one
>> >> >>>>>>>>>> that comes with ParMETIS (that Dorsal builds). I worked around this by
>> >> >>>>>>>>>> configuring DOLFIN with
>> >> >>>>>>>>>> -DMETIS_LIBRARY=/Users/fenicsslave2/Work/FEniCS/lib/libmetis.a (the
>> >> >>>>>>>>>> one that Dorsal installed with ParMETIS).
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> This problem only shows up when PETSc is not found because
>> >> >>>>>>>>>> PETSC_LIBRARIES contains the correct link flags (-L/path/to/libmetis.a
>> >> >>>>>>>>>> -lmetis).
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Johannes
>> >> >>>>>>>>>
>> >> >>>>>>>>> Everything seems to work fine except BOOST_DIR is missing when I run
>> >> >>>>>>>>> the demos.
>> >> >>>>>>>>
>> >> >>>>>>>> /sw (Fink) was the default BOOST_DIR. I have changed this to
>> >> >>>>>>>> /opt/local (MacPorts) now.
>> >> >>>>>>>
>> >> >>>>>>> It works fine when I set BOOST_DIR to /opt/local
>> >> >>>>>>>
>> >> >>>>>>> Should we write BOOST_DIR to dolfin.conf?
>> >> >>>>>>
>> >> >>>>>> Yes, that might be a good idea, however, it might not always be defined.
>> >> >>>>>
>> >> >>>>> Your latest fix seems to have done the trick. It's working fine now.
>> >> >>>>>
>> >> >>>>> So, clean install from Dorsal on Mac OS X without any extra tweaks
>> >> >>>>> seems to work perfectly now, at least for me.
>> >> >>>>>
>> >> >>>>> Anyone else still having trouble?
>> >> >>>>>
>> >> >>>>> I'll try removing some PETSc files next to see if it breaks cleanly.
>> >> >>>>
>> >> >>>> One thing that's still missing on OS X is SLEPc. It has worked before
>> >> >>>> but the test program fails during configuration with the following message:
>> >> >>>>
>> >> >>>> Undefined symbols:
>> >> >>>>  "_ompi_mpi_comm_self"
>> >> >>>
>> >> >>> I'm having a similar problem with PETSc on the Mac buildbot:
>> >> >>>
>> >> >>>   http://fenics.org:8080/builders/dolfin-mac-osx/builds/721/steps/dolfin%20build/logs/CMakeError.log
>> >> >>>
>> >> >>>> I think I remember seeing this in a discussion recently. Is there a
>> >> >>>> solution for this? It has worked before.
>> >> >>>
>> >> >>> One solution will be to set CXX=mpic++ before running cmake. I guess
>> >> >>> the reason it worked before might be that the test was not required to
>> >> >>> work in order for PETSc and SLEPc to be found. This was changed a few
>> >> >>> days back.
>> >> >>
>> >> >> Did we have CXX=mpic++ or similar in the Dorsal platform file for Snow Leopard?
>> >> >
>> >> > We used to have this in dolfin.package:
>> >> >
>> >> > package_specific_setup () {
>> >> >     export CC=mpicc
>> >> >     export CXX=mpic++
>> >> > }
>> >> >
>> >> > Can we add it back? SLEPc works again for me on OS X when I add it.
>> >> >
>> >>
>> >> Best would be to get the MPI flags right in dolfin/CMakeLists.txt. There
>> >> are a few FIXME related the link flags.
>> >>
>> >> If we use the MPI wrappers, we should somehow test for this and then not
>> >> add the MPI compiler flags in dolfin/CMakeLists.txt.
>> >
>> > I don't understand. That seems like a secondary issue, that we would
>> > add flags we don't really need if we are using the MPI wrappers.
>> >
>> > Does anyone know which extra flag is needed to get things to run?
>>
>> I guess that these should do it: -lmpi -lopen-rte -lopen-pal
>
> Is there a simple way to find those flags?

The MPI_LIBRARIES variable from find_package(MPI) should have them.

>> The problem (in my case with PETSc) is that these are missing from
>> PETSC_LIBRARIES so the test program in FindPETSc.cmake fails.
>
> Why does PETSc test fail? The PETSc test is run before the SLEPc test
> so any flags added while running the SLEPc test shouldn't affect
> PETSc.

The PETSc test fails so I never get to the SLEPc test.

> Or we could add the flags to PETSC_LIBRARIES?

Yes, we could use find_package(MPI) in FindPETSc but PETSc might have
been built without MPI.

Johannes