yade-dev team mailing list archive

Thread
Date

Re: Make ccache working

From: François <francois.kneib@xxxxxxxxx>
Date: Thu, 28 Mar 2019 07:40:59 +0100
Cc: Yade developers <yade-dev@xxxxxxxxxxxxxxxxxxx>
In-reply-to: <CANFfKpE5o0xySo5yFtA5feW50Fc7VdGeHvpGR6rbTbHTam7XGw@mail.gmail.com>

Here attached is the full cmake output when I did:

> rm -r *
> export CC="ccache /usr/bin/gcc"
> export CXX="ccache /usr/bin/g++"
> cmake -DCMAKE_INSTALL_PREFIX=../install ../trunk
>


Le jeu. 28 mars 2019 à 06:48, Bruno Chareyre <bruno.chareyre@xxxxxxxxxxxxxxx>
a écrit :

> Tu peux ajouter la sortie de cmake a ton message? Pas dispo ce matin.
> B
>
> Le jeu. 28 mars. 2019 00:58, François <francois.kneib@xxxxxxxxx> a écrit :
>
>> Thanks for your help. I'm on kubuntu 18.04 (cmake 3.10.2), and maybe our
>> versions mismatch is at the origin of the issue (I finally found a
>> solution, see the bottom of this message).
>> By "installing" ccache I meant in fact "enable/use" ccache, I also
>> installed it with "apt install ccache". Of course, I always start from an
>> empty build dir.
>>
>> Now I restarted everything with a clean $PATH, reverting my changes.
>> Using export CC and CXX doesn't work:
>>
>>> export CC="ccache gcc"
>>> export CXX="ccache g++"
>>> cmake -DCMAKE_INSTALL_PREFIX=../install ../trunk
>>> [...]
>>> *ccache: error: Could not find compiler "g++" in PATH*
>>>
>> Somebody had the same issue here:
>> https://github.com/bazelbuild/bazel/issues/3642 so I tried
>> unsuccessfully :
>>
>>> rm -r *
>>> export CC="ccache /usr/bin/gcc"
>>> export CXX="ccache /usr/bin/g++"
>>> cmake -DCMAKE_INSTALL_PREFIX=../install ../trunk
>>> [...]
>>> */usr/bin/ccache: invalid option -- 'd'*
>>>
>> I didn't found exactly why, but somewhere the "-d" option is passed
>> directly to ccache (instead of gcc/g++ I assume), but not in your 16.04
>> system.
>>
>> So I tried two methods to use $PATH instead of $CXX and $CC:
>> - the one in ccache documentation, with symlinks :
>> https://ccache.samba.org/manual/latest.html#_run_modes
>> - the one we can find everywhere with export PATH="/usr/lib/ccache:$PATH"
>> But the result is finally the same as above.
>>
>> *OUTCOME*:
>> I feel a bit confused as I spent some time to make it work to finally see
>> that cmake handles ccache since version 3.4:
>> https://cmake.org/cmake/help/v3.6/release/3.4.html#variables
>> Without environment variables nor symlinks, the solution basically is to
>> run cmake with "*-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
>> -DCMAKE_C_COMPILER_LAUNCHER=ccache*"
>> Now ccache is activated, after a successful build I have 720.9 MB in
>> cache, yade full build in 30s yeepee :-D
>> This solution should also work for ubuntu 16.04 as it carries cmake 3.5,
>> but not for ubuntu 14.04.
>>
>> NB: Interestingly, compiling [yade with python 3] for the first time with
>> the full ccache of [yade with python 2.7] took about 5 mins against about
>> 30min for a full uncached build.
>>
>> Cheers,
>> François
>>
>>
>>
>>
>> Le mer. 27 mars 2019 à 20:40, Bruno Chareyre <
>> bruno.chareyre@xxxxxxxxxxxxxxx> a écrit :
>>
>>> I checked your ubuntu 12.04 page again. That's actually about hiding
>>> ccache behind normal gcc invocation, and I kind-of dislike the idea.
>>> Explicit call to «ccache g++» sounds cleaner to me and it is compatible
>>> with mpicxx (for ubuntu 16.04 at least).
>>> Better undo what you have done until now in terms of ccache hacks and
>>> stick to yade doc's method if you are unsure.
>>>
>>> «ccache -s» also reports unsupported options on my ubuntu 16.04 but a
>>> full rebuild takes 40sec regardless.
>>>
>>> Bruno
>>>
>>> Le mer. 27 mars. 2019 19:46, Bruno Chareyre <
>>> bruno.chareyre@xxxxxxxxxxxxxxx> a écrit :
>>>
>>>> Also, did you start from fresh build after exporting CXX?
>>>> I mean:
>>>> rm -rf ./*
>>>> export ...
>>>> cmake [options]
>>>> make -jN
>>>>
>>>> Bruno
>>>>
>>>> Le mer. 27 mars. 2019 18:33, Bruno Chareyre <
>>>> bruno.chareyre@xxxxxxxxxxxxxxx> a écrit :
>>>>
>>>>> Hi François,
>>>>>
>>>>> Please remember to give your linux version.
>>>>>
>>>>> On Wed, 27 Mar 2019 at 17:44, François <francois.kneib@xxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> export CXX=ccache g++
>>>>>>
>>>>>>> bash: export: « g++ » : identifiant non valable
>>>>>>>
>>>>>> Using
>>>>>>
>>>>>>> export CXX='ccache g++'
>>>>>>
>>>>>> works
>>>>>>
>>>>>
>>>>> Thanks for pointing that out. I forgot the " indeed, I'll fix it.
>>>>> It would be useful to turn your bash to english for pasting errors
>>>>> here (also much more efficient when searching online solutions).
>>>>>
>>>>>
>>>>>
>>>>>> but it will result in cmake fail while checking cc compiler.
>>>>>> Finally, I "installed" ccache thanks to the first answer here:
>>>>>> https://askubuntu.com/questions/470545/how-do-i-set-up-ccache and it
>>>>>> *kinda* works.
>>>>>>
>>>>>
>>>>> Arguably, using ccache is assuming that it is installed. ;)
>>>>> Not sure why you needed that ubuntu 12.04 forum, as far as I remember
>>>>> I only had to "apt-get install ccache".
>>>>>
>>>>>
>>>>>>
>>>>>>    - the second one, once the install is ok and cmake uses ccache,
>>>>>>    is that I only get "unsupported compiler option" when I try *ccache
>>>>>>    -s* after a full yade compilation. It results in a almost-empty
>>>>>>    cache:
>>>>>>
>>>>>>
>>>>> Does compilation work?
>>>>> Could you send cmake output, and some sample of "make VERBOSE=1"?
>>>>>
>>>>> I have this:
>>>>>
>>>>> *[ 39%] Building CXX object
>>>>> CMakeFiles/yade.dir/pkg/dem/Ig2_Box_Sphere_ScGeom.cpp.o/usr/bin/ccache
>>>>> g++  -DBOOST_MATH_DISABLE_FLOAT128=1 -DNDE...*
>>>>>
>>>>> Cheers
>>>>>
>>>>> Bruno
>>>>>
>>>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~yade-dev
>>> Post to     : yade-dev@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~yade-dev
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~yade-dev
>> Post to     : yade-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~yade-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>

-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/ccache
-- Check for working C compiler: /usr/bin/ccache -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/ccache
-- Check for working CXX compiler: /usr/bin/ccache -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/bin/python3 (found version "3.6.7") 
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
qmake: could not exec '/usr/lib/x86_64-linux-gnu/qt4/bin/qmake': No such file or directory
-- Found unsuitable Qt version "" from NOTFOUND
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1") 
-- Version is set to 2019-03-27.git-accdc52
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so   
-- GTS using gts-config /usr/bin/gts-config
-- Using GTS from /usr
-- Found GL2PS: /usr/lib/x86_64-linux-gnu/libgl2ps.so  
-- Found CGAL: /usr/include, /usr/lib/x86_64-linux-gnu/libCGAL.so
-- Found NumPy: version "1.16.2" /home/francois/.local/lib/python3.6/site-packages/numpy/core/include
-- Found Loki: /usr/include  
/usr/bin/ccache: invalid option -- 'd'
Usage:
    ccache [options]
    ccache compiler [compiler options]
    compiler [compiler options]          (via symbolic link)

Options:
    -c, --cleanup         delete old files and recalculate size counters
                          (normally not needed as this is done automatically)
    -C, --clear           clear the cache completely (except configuration)
    -F, --max-files=N     set maximum number of files in cache to N (use 0 for
                          no limit)
    -M, --max-size=SIZE   set maximum size of cache to SIZE (use 0 for no
                          limit); available suffixes: k, M, G, T (decimal) and
                          Ki, Mi, Gi, Ti (binary); default suffix: G
    -o, --set-config=K=V  set configuration key K to value V
    -p, --print-config    print current configuration options
    -s, --show-stats      show statistics summary
    -z, --zero-stats      zero statistics counters

    -h, --help            print this help text
    -V, --version         print version and copyright information

See also <https://ccache.samba.org>.
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   python
--   thread
--   filesystem
--   iostreams
--   regex
--   serialization
--   system
--   date_time
--   chrono
--   atomic
--   Boost_VERSION: 106501
--   Boost_LIB_VERSION: 1_65_1
--   Boost_INCLUDE_DIRS: /usr/include
--   Boost_LIBRARIES: /usr/lib/x86_64-linux-gnu/libboost_python.so/usr/lib/x86_64-linux-gnu/libboost_thread.so/usr/lib/x86_64-linux-gnu/libboost_filesystem.so/usr/lib/x86_64-linux-gnu/libboost_iostreams.so/usr/lib/x86_64-linux-gnu/libboost_regex.so/usr/lib/x86_64-linux-gnu/libboost_serialization.so/usr/lib/x86_64-linux-gnu/libboost_system.so/usr/lib/x86_64-linux-gnu/libboost_date_time.so/usr/lib/x86_64-linux-gnu/libboost_chrono.so/usr/lib/x86_64-linux-gnu/libboost_atomic.so/usr/lib/x86_64-linux-gnu/libpthread.so
-- Found Eigen3: /usr/include/eigen3 (Required is at least version "2.91.0") 
-- Found BZip2: /usr/lib/x86_64-linux-gnu/libbz2.so (found version "1.0.6") 
-- Looking for BZ2_bzCompressInit
-- Looking for BZ2_bzCompressInit - found
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11") 
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found version "2.7.15rc1") 
-- Found Eigen3, version: 3.3.4
-- Disable vectorization
-- The imported target "vtkRenderingPythonTkWidgets" references the file
   "/usr/lib/x86_64-linux-gnu/libvtkRenderingPythonTkWidgets.so"
but this file does not exist.  Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
   "/usr/lib/cmake/vtk-6.3/VTKTargets.cmake"
but not all the files it references.

-- The imported target "vtk" references the file
   "/usr/bin/vtk"
but this file does not exist.  Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
   "/usr/lib/cmake/vtk-6.3/VTKTargets.cmake"
but not all the files it references.

-- Found VTK
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP
-- GTS using gts-config /usr/bin/gts-config
-- Using GTS from /usr
-- Checking for one of the modules 'glib-2.0'
-- Found GLib2: glib-2.0 /usr/include/glib-2.0;/usr/lib/x86_64-linux-gnu/glib-2.0/include
-- Looking for include file glib/gregex.h
-- Looking for include file glib/gregex.h - not found
-- Found GTS
-- Found GLUT: /usr/lib/x86_64-linux-gnu/libglut.so  
-- USE QT5
-- Found QGLVIEWER-qt5: /usr/include/QGLViewer  
-- Found GUI-Qt5-LIBS
-- GMP libs: /usr/lib/x86_64-linux-gnu/libgmp.so /usr/lib/x86_64-linux-gnu/libgmpxx.so
-- Found GMP: /usr/include/x86_64-linux-gnu  
-- GMP libs: /usr/lib/x86_64-linux-gnu/libgmp.so /usr/lib/x86_64-linux-gnu/libgmpxx.so
-- Found CGAL
-- SuiteSparse version 5.1.2 found, CHOLMOD direct solver for CPU activated.
-- Found Cholmod: /usr/lib/x86_64-linux-gnu/libcholmod.so  
-- Found OpenBlas: /usr/lib/x86_64-linux-gnu/libopenblas.so  
-- Found Metis: /usr/include  
-- Found Cholmod in /usr/lib/x86_64-linux-gnu/libcholmod.so
-- Found OpenBlas in /usr/lib/x86_64-linux-gnu/libopenblas.so
-- Found Metis in /usr/lib/x86_64-linux-gnu/libmetis.so
-- Found GL2PS
LBMFLOW is still experimental, building and running LBM engine are at your own risk!
Yade will be installed to /home/francois/Documents/Micro-entreprise/projets/yade-python3/install
-- Suffix is set to -2019-03-27.git-accdc52
-- LIBRARY_OUTPUT_PATH is set to lib/x86_64-linux-gnu
-- runtimePREFIX is set to /home/francois/Documents/Micro-entreprise/projets/yade-python3/install
-- Could NOT find gts (missing: PY_gts) 
-- Use embedded version of gts. Please, consider installing the corresponding package
-- Found minieigen: /usr/lib/python3/dist-packages/minieigen.cpython-36m-x86_64-linux-gnu.so  
-- Found tkinter: /usr/lib/python3.6/tkinter  
-- VTK version >5 and <8 is found
-- ===========================================================
-- Yade configured with following features: Odeint VTK OpenMP GTS GUI-Qt5 CGAL PFVFLOW LINSOLV TWOPHASEFLOW GL2PS LBMFLOW
-- Disabled features: CHOLMOD_GPU SPH DEFORM LIQMIGRATION MASK_ARBITRARY THERMAL PROFILING PotentialParticles PotentialBlocks
-- Optimized build
-- ===========================================================
-- Configuring done
-- Generating done
-- Build files have been written to: /home/francois/Documents/Micro-entreprise/projets/yade-python3/build-trunk

Follow ups

Re: Make ccache working
From: Bruno Chareyre, 2019-03-28

References

Make ccache working
From: François, 2019-03-27
Re: Make ccache working
From: Bruno Chareyre, 2019-03-27
Re: Make ccache working
From: Bruno Chareyre, 2019-03-27
Re: Make ccache working
From: Bruno Chareyre, 2019-03-27
Re: Make ccache working
From: François, 2019-03-27
Re: Make ccache working
From: Bruno Chareyre, 2019-03-28