← Back to team overview

dolfin team mailing list archive

Re: Extending DOLFIN CMake scripts for CUDA interoperability

 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 19/01/11 09:33, Anders Logg wrote:
> On Tue, Jan 18, 2011 at 06:54:32PM +0000, Florian Rathgeber wrote:
> Hi,
> 
> I also finally managed to trace it down to the quotes in the
> PACKAGE_VERSION define, but forgot to report that back right away, sorry
> about that.
> 
> The reason behind it is some magic FindCUDA.cmake tries to do with
> quotes in compiler flags which is apparently broken. I reported the bug
> to the FindCUDA.cmake authors.
> 
>> FindFoo scripts may sometimes be broken, even those scripts that are
>> shipped with CMake or as part of the libraries they are searching for
>> so sometimes writing a new script is a better solution. If you can
>> write a better script we can just include it in cmake/modules.

The FindCUDA script does way more than merely finding CUDA components.
It does all sorts of magic to enable compilation of .cu files with nvcc.
I wouldn't dare trying to re-implement (let alone better) and it's quite
mature in general (apart from this particular bug). It's a bit nasty in
that it creates 2 cmake files for each .cu source, but this has to do
with peculiarities of nvcc again.

I did however place the current svn version of it in cmake/modules since
it contains some fixes for CUDA 3.2 that are not in the version shipped
with CMake 2.8.2.

However I'm quite baffled why setting CMake cache entries on the command
line doesn't work as it should (see below).

Florian

>> --
>> Anders
> 
> 
> The error you are seeing then is because I committed a broken
> intermediate version so you can see the error. I'll push a new version
> once I worked around the other problems I came across.
> 
> To follow up on the story of building DOLFIN with CUDA support and a bit
> of background:
> 
> C for CUDA is basically an extension to C (C++ is mostly supported now).
> The NVIDIA compiler nvcc splits CUDA source files in host and device
> code parts, takes care of the device code and passes host C++ code to
> the gcc host compiler. There are notorious incompatibilities between
> certain nvcc and gcc versions. As of now (nvcc 3.2) gcc up to 4.4 is
> supported in general, but there are problems with certain parts of the
> STL (in particular pretty much everything from tr1) and even more so boost.
> 
> I posted some of the issues I encountered and workarounds in my replies
> to this thread: http://forums.nvidia.com/index.php?showtopic=182890
> 
> So finally I got my code to build but still need to verify it actually
> works.
> 
> One thing I'm still struggling is how to set nvcc flags from the
> cmake.local script. There is a CMake cache variable CUDA_NVCC_FLAGS for
> that, but setting it from the command line
> (-DCUDA_NVCC_FLAGS="--ptxas-options=-v;--compiler-options=-D__builtin_isnormal=isnormal;--compiler-options=-DBOOST_UNORDERED_NO_HAS_MOVE_ASSIGN")
> doesn't have any effect. I don't want to hard code these in the
> CMakeLists.txt. Something as simple as -DCUDA_NVCC_FLAGS="-G" works though.
> 
> Florian
> 
> On 18/01/11 13:42, Johannes Ring wrote:
>>>> On Tue, Jan 18, 2011 at 12:55 PM, Anders Logg <logg@xxxxxxxxx> wrote:
>>>>> On Tue, Jan 18, 2011 at 09:26:06AM +0100, Johannes Ring wrote:
>>>>>> On Mon, Jan 17, 2011 at 10:00 PM, Johan Hake <johan.hake@xxxxxxxxx> wrote:
>>>>>>> It looks like changing:
>>>>>>>
>>>>>>>  add_definitions(-DPACKAGE_VERSION="${DOLFIN_VERSION}")
>>>>>>>
>>>>>>> to
>>>>>>>
>>>>>>>  add_definitions(-DPACKAGE_VERSION=${DOLFIN_VERSION})
>>>>>>>
>>>>>>> Fixed it. PACKAGE_VERSION got added to the defines in the call to nvcc and the
>>>>>>> escaped quotes screwed things up.
>>>>>>>
>>>>>>> Johannes: Are the quotes needed for something special? It compiles alright
>>>>>>> without the quotes.
>>>>>>
>>>>>> It does not work for me when I remove the quotes:
>>>>>>
>>>>>> [ 50%] Building CXX object dolfin/CMakeFiles/dolfin.dir/main/init.cpp.o
>>>>>> /home/johannr/src/bzr/dolfin/dolfin/main/init.cpp:15:53: error: too
>>>>>> many decimal points in number
>>>>>> /home/johannr/src/bzr/dolfin/dolfin/main/init.cpp: In function void
>>>>>> dolfin::dolfin_init(int, char**):
>>>>>> /home/johannr/src/bzr/dolfin/dolfin/main/init.cpp:15: error: expected
>>>>>> primary-expression before ) token
>>>>>> make[2]: *** [dolfin/CMakeFiles/dolfin.dir/main/init.cpp.o] Error 1
>>>>>> make[1]: *** [dolfin/CMakeFiles/dolfin.dir/all] Error 2
>>>>>> make: *** [all] Error 2
>>>>>>
>>>>>> However, I don't think the -DPACKAGE_VERSION flag is really needed. It
>>>>>> only seems to be used in dolfin/common/constants.h for specifying the
>>>>>> DOLFIN_VERSION. We can use CMake's configure_file command instead to
>>>>>> set this value. How does that sound?
>>>>>
>>>>> Good.
>>>>
>>>> Unfortunately, this turned out to be more involved than I thought so I
>>>> went back to use the -DPACKAGE_VERSION flag. However, it seems to work
>>>> better when the flag is added later, that is, after the call to
>>>> find_package(CUDA). I couldn't really test this since my computer
>>>> doesn't have a NVIDIA GPU but at least I managed to get the nvcc
>>>> compiler running. It failed with a message complaining about that the
>>>> file gpu_util.h could not be found. I guess this file comes with the
>>>> NVIDIA drivers or something. Anyway, the DOLFIN development repository
>>>> is now updated with this fix.
>>>>
>>>> Johannes
>>>>
>>>>>
>>>>>
>>>>>> Johannes
>>>>>>
>>>>>>> Johan
>>>>>>>
>>>>>>> On Monday January 17 2011 09:41:26 Johan Hake wrote:
>>>>>>>> Florian!
>>>>>>>>
>>>>>>>> I am no CMake expert nor have I ever compiled any CUDA code, but it looks
>>>>>>>> like something fishy happens when you add your flags to the CUDA code
>>>>>>>> generation step. Have you set it up correctly? I also get the erronious
>>>>>>>> output from CMake for the nvcc call.
>>>>>>>>
>>>>>>>> What CMake command is used to set these flags? It almost seems like the
>>>>>>>> eronious flags are set automatic? At least I cannot see that you put
>>>>>>>> PACKAGE_VERSION to the NVCC flags anywhere. If they are automatically set,
>>>>>>>> maybe there is a way to set them using one of the commands from
>>>>>>>> FindCUDA.cmake?
>>>>>>>>
>>>>>>>> Johan
>>>>>>>>
>>>>>>>> On Friday January 14 2011 12:07:50 Florian Rathgeber wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I'm currently trying to get my DOLFIN GPU code in shape but haven't
>>>>>>>>> quite succeeded integrating it with DOLFIN's CMake build.
>>>>>>>>>
>>>>>>>>> Now the FindCUDA.cmake script tries to propagate host compiler flags and
>>>>>>>>> defines to the NVIDIA compiler nvcc but apparently messes up so I get a
>>>>>>>>>
>>>>>>>>> compiler invocation looking like this (only the relevant part):
>>>>>>>>>  -DPACKAGE_VERSION= 0.9.9+\" -Ddolfin_gpu_EXPORTS\" -Xcompiler
>>>>>>>>>
>>>>>>>>> ,\"-fPIC\",\"-O2\",\"-g\"
>>>>>>>>>
>>>>>>>>> Note the odd quoting and also the space after -DPACKAGE_VERSION=
>>>>>>>>>
>>>>>>>>> This causes nvcc to assume this is a target and die:
>>>>>>>>> nvcc fatal   : A single input file is required for a non-link phase when
>>>>>>>>> an outputfile is specified
>>>>>>>>>
>>>>>>>>> The code is at lp:~florian-rathgeber/dolfin/gpu-backend
>>>>>>>>>
>>>>>>>>> The relevant section from dolfin/CMakeLists.txt (after all the other
>>>>>>>>> packages)
>>>>>>>>> <<<
>>>>>>>>> # CUDA
>>>>>>>>> SET(DOLFIN_GPU_LIBRARY "")
>>>>>>>>> if (DOLFIN_ENABLE_CUDA AND CUDA_FOUND)
>>>>>>>>>
>>>>>>>>>   list(APPEND DOLFIN_CXX_DEFINITIONS "-DHAS_CUDA")
>>>>>>>>>   list(APPEND DOLFIN_INCLUDE_DIRECTORIES ${CUDA_INCLUDE_DIRS})
>>>>>>>>>   # Add CUDA sources (*.cu) only if CUDA is enabled
>>>>>>>>>   foreach(DIR ${DOLFIN_DIRS})
>>>>>>>>>
>>>>>>>>>     file(GLOB _CU_SOURCES ${DIR}/*.cu)
>>>>>>>>>     list(APPEND CU_SOURCES ${_CU_SOURCES})
>>>>>>>>>
>>>>>>>>>   endforeach()
>>>>>>>>>   CUDA_ADD_LIBRARY(dolfin_gpu ${CU_SOURCES} SHARED)
>>>>>>>>>   SET(DOLFIN_GPU_LIBRARY dolfin_gpu)
>>>>>>>>>   list(APPEND DOLFIN_TARGET_LINK_LIBRARIES ${CUDA_LIBRARIES}
>>>>>>>>>
>>>>>>>>> ${DOLFIN_GPU_LIBRARY})
>>>>>>>>> endif()
>>>>>>>>>
>>>>>>>>> # Cusp
>>>>>>>>> if (DOLFIN_ENABLE_CUSP AND CUSP_FOUND)
>>>>>>>>>
>>>>>>>>>   list(APPEND DOLFIN_CXX_DEFINITIONS "-DHAS_CUSP")
>>>>>>>>>   list(APPEND DOLFIN_INCLUDE_DIRECTORIES ${THRUST_INCLUDE_DIRS}
>>>>>>>>>
>>>>>>>>>     ${CUSP_INCLUDE_DIRS})
>>>>>>>>>
>>>>>>>>> endif()
>>>>>>>>> <<<
>>>>>>>>>
>>>>>>>>> In CMakeLists.txt I check for the packages like this:
>>>>>>>>> <<<
>>>>>>>>> # Check for CUDA
>>>>>>>>> if (DOLFIN_ENABLE_CUDA)
>>>>>>>>>
>>>>>>>>>   find_package(CUDA)
>>>>>>>>>
>>>>>>>>> endif()
>>>>>>>>>
>>>>>>>>> # Check for Cusp
>>>>>>>>> if (DOLFIN_ENABLE_CUSP)
>>>>>>>>>
>>>>>>>>>   find_package(Cusp)
>>>>>>>>>
>>>>>>>>> endif()
>>>>>>>>> <<<
>>>>>>>>> FindCUDA is the stock script from CMake 2.8, FindCusp is a simple script
>>>>>>>>> I wrote which lives in cmake/modules
>>>>>>>>>
>>>>>>>>> I'd be happy about any hints and tips. Garth, afair you wanted to look
>>>>>>>>> into this at some point now that the new build system is in place?
>>>>>>>>>
>>>>>>>>> Florian
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Mailing list: https://launchpad.net/~dolfin
>>>>>>>> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
>>>>>>>> Unsubscribe : https://launchpad.net/~dolfin
>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dolfin
>>>>>>> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dolfin
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~dolfin
>>>>>> Post to     : dolfin@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~dolfin
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk029E0ACgkQ8Z6llsctAxZ7MACgkLh/iA4qCRRtgO7hKHy8NVY7
Np0AoM11GgDDPQV757v8ABAmIlpJXWD2
=ku85
-----END PGP SIGNATURE-----

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature


Follow ups

References