← Back to team overview

dolfin team mailing list archive

Re: Extending DOLFIN CMake scripts for CUDA interoperability

 

On Tue, Jan 18, 2011 at 06:54:32PM +0000, Florian Rathgeber wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> I also finally managed to trace it down to the quotes in the
> PACKAGE_VERSION define, but forgot to report that back right away, sorry
> about that.
>
> The reason behind it is some magic FindCUDA.cmake tries to do with
> quotes in compiler flags which is apparently broken. I reported the bug
> to the FindCUDA.cmake authors.

FindFoo scripts may sometimes be broken, even those scripts that are
shipped with CMake or as part of the libraries they are searching for
so sometimes writing a new script is a better solution. If you can
write a better script we can just include it in cmake/modules.

--
Anders


> The error you are seeing then is because I committed a broken
> intermediate version so you can see the error. I'll push a new version
> once I worked around the other problems I came across.
>
> To follow up on the story of building DOLFIN with CUDA support and a bit
> of background:
>
> C for CUDA is basically an extension to C (C++ is mostly supported now).
> The NVIDIA compiler nvcc splits CUDA source files in host and device
> code parts, takes care of the device code and passes host C++ code to
> the gcc host compiler. There are notorious incompatibilities between
> certain nvcc and gcc versions. As of now (nvcc 3.2) gcc up to 4.4 is
> supported in general, but there are problems with certain parts of the
> STL (in particular pretty much everything from tr1) and even more so boost.
>
> I posted some of the issues I encountered and workarounds in my replies
> to this thread: http://forums.nvidia.com/index.php?showtopic=182890
>
> So finally I got my code to build but still need to verify it actually
> works.
>
> One thing I'm still struggling is how to set nvcc flags from the
> cmake.local script. There is a CMake cache variable CUDA_NVCC_FLAGS for
> that, but setting it from the command line
> (-DCUDA_NVCC_FLAGS="--ptxas-options=-v;--compiler-options=-D__builtin_isnormal=isnormal;--compiler-options=-DBOOST_UNORDERED_NO_HAS_MOVE_ASSIGN")
> doesn't have any effect. I don't want to hard code these in the
> CMakeLists.txt. Something as simple as -DCUDA_NVCC_FLAGS="-G" works though.
>
> Florian
>
> On 18/01/11 13:42, Johannes Ring wrote:
> > On Tue, Jan 18, 2011 at 12:55 PM, Anders Logg <logg@xxxxxxxxx> wrote:
> >> On Tue, Jan 18, 2011 at 09:26:06AM +0100, Johannes Ring wrote:
> >>> On Mon, Jan 17, 2011 at 10:00 PM, Johan Hake <johan.hake@xxxxxxxxx> wrote:
> >>>> It looks like changing:
> >>>>
> >>>>  add_definitions(-DPACKAGE_VERSION="${DOLFIN_VERSION}")
> >>>>
> >>>> to
> >>>>
> >>>>  add_definitions(-DPACKAGE_VERSION=${DOLFIN_VERSION})
> >>>>
> >>>> Fixed it. PACKAGE_VERSION got added to the defines in the call to nvcc and the
> >>>> escaped quotes screwed things up.
> >>>>
> >>>> Johannes: Are the quotes needed for something special? It compiles alright
> >>>> without the quotes.
> >>>
> >>> It does not work for me when I remove the quotes:
> >>>
> >>> [ 50%] Building CXX object dolfin/CMakeFiles/dolfin.dir/main/init.cpp.o
> >>> /home/johannr/src/bzr/dolfin/dolfin/main/init.cpp:15:53: error: too
> >>> many decimal points in number
> >>> /home/johannr/src/bzr/dolfin/dolfin/main/init.cpp: In function ‘void
> >>> dolfin::dolfin_init(int, char**)’:
> >>> /home/johannr/src/bzr/dolfin/dolfin/main/init.cpp:15: error: expected
> >>> primary-expression before ‘)’ token
> >>> make[2]: *** [dolfin/CMakeFiles/dolfin.dir/main/init.cpp.o] Error 1
> >>> make[1]: *** [dolfin/CMakeFiles/dolfin.dir/all] Error 2
> >>> make: *** [all] Error 2
> >>>
> >>> However, I don't think the -DPACKAGE_VERSION flag is really needed. It
> >>> only seems to be used in dolfin/common/constants.h for specifying the
> >>> DOLFIN_VERSION. We can use CMake's configure_file command instead to
> >>> set this value. How does that sound?
> >>
> >> Good.
> >
> > Unfortunately, this turned out to be more involved than I thought so I
> > went back to use the -DPACKAGE_VERSION flag. However, it seems to work
> > better when the flag is added later, that is, after the call to
> > find_package(CUDA). I couldn't really test this since my computer
> > doesn't have a NVIDIA GPU but at least I managed to get the nvcc
> > compiler running. It failed with a message complaining about that the
> > file gpu_util.h could not be found. I guess this file comes with the
> > NVIDIA drivers or something. Anyway, the DOLFIN development repository
> > is now updated with this fix.
> >
> > Johannes
> >
> >>
> >>
> >>> Johannes
> >>>
> >>>> Johan
> >>>>
> >>>> On Monday January 17 2011 09:41:26 Johan Hake wrote:
> >>>>> Florian!
> >>>>>
> >>>>> I am no CMake expert nor have I ever compiled any CUDA code, but it looks
> >>>>> like something fishy happens when you add your flags to the CUDA code
> >>>>> generation step. Have you set it up correctly? I also get the erronious
> >>>>> output from CMake for the nvcc call.
> >>>>>
> >>>>> What CMake command is used to set these flags? It almost seems like the
> >>>>> eronious flags are set automatic? At least I cannot see that you put
> >>>>> PACKAGE_VERSION to the NVCC flags anywhere. If they are automatically set,
> >>>>> maybe there is a way to set them using one of the commands from
> >>>>> FindCUDA.cmake?
> >>>>>
> >>>>> Johan
> >>>>>
> >>>>> On Friday January 14 2011 12:07:50 Florian Rathgeber wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> I'm currently trying to get my DOLFIN GPU code in shape but haven't
> >>>>>> quite succeeded integrating it with DOLFIN's CMake build.
> >>>>>>
> >>>>>> Now the FindCUDA.cmake script tries to propagate host compiler flags and
> >>>>>> defines to the NVIDIA compiler nvcc but apparently messes up so I get a
> >>>>>>
> >>>>>> compiler invocation looking like this (only the relevant part):
> >>>>>>  -DPACKAGE_VERSION= 0.9.9+\" -Ddolfin_gpu_EXPORTS\" -Xcompiler
> >>>>>>
> >>>>>> ,\"-fPIC\",\"-O2\",\"-g\"
> >>>>>>
> >>>>>> Note the odd quoting and also the space after -DPACKAGE_VERSION=
> >>>>>>
> >>>>>> This causes nvcc to assume this is a target and die:
> >>>>>> nvcc fatal   : A single input file is required for a non-link phase when
> >>>>>> an outputfile is specified
> >>>>>>
> >>>>>> The code is at lp:~florian-rathgeber/dolfin/gpu-backend
> >>>>>>
> >>>>>> The relevant section from dolfin/CMakeLists.txt (after all the other
> >>>>>> packages)
> >>>>>> <<<
> >>>>>> # CUDA
> >>>>>> SET(DOLFIN_GPU_LIBRARY "")
> >>>>>> if (DOLFIN_ENABLE_CUDA AND CUDA_FOUND)
> >>>>>>
> >>>>>>   list(APPEND DOLFIN_CXX_DEFINITIONS "-DHAS_CUDA")
> >>>>>>   list(APPEND DOLFIN_INCLUDE_DIRECTORIES ${CUDA_INCLUDE_DIRS})
> >>>>>>   # Add CUDA sources (*.cu) only if CUDA is enabled
> >>>>>>   foreach(DIR ${DOLFIN_DIRS})
> >>>>>>
> >>>>>>     file(GLOB _CU_SOURCES ${DIR}/*.cu)
> >>>>>>     list(APPEND CU_SOURCES ${_CU_SOURCES})
> >>>>>>
> >>>>>>   endforeach()
> >>>>>>   CUDA_ADD_LIBRARY(dolfin_gpu ${CU_SOURCES} SHARED)
> >>>>>>   SET(DOLFIN_GPU_LIBRARY dolfin_gpu)
> >>>>>>   list(APPEND DOLFIN_TARGET_LINK_LIBRARIES ${CUDA_LIBRARIES}
> >>>>>>
> >>>>>> ${DOLFIN_GPU_LIBRARY})
> >>>>>> endif()
> >>>>>>
> >>>>>> # Cusp
> >>>>>> if (DOLFIN_ENABLE_CUSP AND CUSP_FOUND)
> >>>>>>
> >>>>>>   list(APPEND DOLFIN_CXX_DEFINITIONS "-DHAS_CUSP")
> >>>>>>   list(APPEND DOLFIN_INCLUDE_DIRECTORIES ${THRUST_INCLUDE_DIRS}
> >>>>>>
> >>>>>>     ${CUSP_INCLUDE_DIRS})
> >>>>>>
> >>>>>> endif()
> >>>>>> <<<
> >>>>>>
> >>>>>> In CMakeLists.txt I check for the packages like this:
> >>>>>> <<<
> >>>>>> # Check for CUDA
> >>>>>> if (DOLFIN_ENABLE_CUDA)
> >>>>>>
> >>>>>>   find_package(CUDA)
> >>>>>>
> >>>>>> endif()
> >>>>>>
> >>>>>> # Check for Cusp
> >>>>>> if (DOLFIN_ENABLE_CUSP)
> >>>>>>
> >>>>>>   find_package(Cusp)
> >>>>>>
> >>>>>> endif()
> >>>>>> <<<
> >>>>>> FindCUDA is the stock script from CMake 2.8, FindCusp is a simple script
> >>>>>> I wrote which lives in cmake/modules
> >>>>>>
> >>>>>> I'd be happy about any hints and tips. Garth, afair you wanted to look
> >>>>>> into this at some point now that the new build system is in place?
> >>>>>>
> >>>>>> Florian
> >>>>>
> >>>>> _______________________________________________
> >>>>> Mailing list: https://launchpad.net/~dolfin
> >>>>> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
> >>>>> Unsubscribe : https://launchpad.net/~dolfin
> >>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>
> >>>> _______________________________________________
> >>>> Mailing list: https://launchpad.net/~dolfin
> >>>> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
> >>>> Unsubscribe : https://launchpad.net/~dolfin
> >>>> More help   : https://help.launchpad.net/ListHelp
> >>>>
> >>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~dolfin
> >>> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~dolfin
> >>> More help   : https://help.launchpad.net/ListHelp
> >>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAk014egACgkQ8Z6llsctAxYILgCfZo9myVI8WJwbWdwwF6myWBuN
> 1UwAnAptSR6+hTkXeQX5cXMI+UOLfFoE
> =VLrZ
> -----END PGP SIGNATURE-----
>





Follow ups

References