kicad-developers team mailing list archive

[Lorenzo Marcantonio <l.marcantonio@xxxxxxxxxxxx>]

On Wed, Oct 22, 2014 at 08:16:07AM -0700, Henner Zeller wrote:
> There are a lot of dependencies (=timestamps) to be checked, so having
> a solid state disk can probably speed up things dramatically.

Also, with enough RAM (even 4GB should be quite enough) the stat
operation would be done from the cache (it only needs to read the inode
itself, not the file).

> Also, for incremental re-compile after a make clean, you might want to
> look into ccache, that allows to locally cache compiler outputs (not
> sure if that is available on Windows proper, but an internet search
> suggests as such). ccache speeds up a (make clean ; make) to abot 3:20
> minutes. It looks like there is a lot of time wasted in partially
> re-building and re-packing the boos library, but I think that is a
> separate discussion (apparently it makes sense on some systems)

The boost thing is a one-time nuisance and we are talking about
incremental builds... so even ccache wouldn't be useful... unless
dependencies are wrong:P

> Compiling things in parallel is always a good idea, and as others
> pointed out, the -j option is good for that. In my case, I just set
> the environment variable
> MAKEFLAGS to '-j6' to make sure I never forget that.
> 
> (FWIW, just typing 'make' on an already build KiCAD (I think this is
> what you are doing) takes about 3.6 seconds on my ~3 year old Linux
> machine (4 core i5 2500k, probably similar to yours) - I do have a
> solid state disk, but can't compare what difference that makes)

OK, since we are benchmarking, here are my results (on my
sucky-but-nearly-indestructible panasonic CF-30, 2 cores, 5400 RPM HDD,
3GB RAM)

- Cold cache:
  make -j6  15,31s user 2,27s system 126% cpu 13,902 total
- Hot cache:
  make -j6  15,62s user 2,05s system 168% cpu 10,517 total
- Hot cache, not parallel:
  make  13,75s user 1,88s system 98% cpu 15,886 total

So, it seems that there is indeed a *lot* of filesystem overhead
involved (running ext4 BTW)...

Solution of the mystery: cmake adds as dependency *each single* wx
include file and each boost file used. That's something over 100K files,
for building pcbnew alone :P

After epurating the depend.{internal,make} files; the result is:

make  1,13s user 0,33s system 92% cpu 1,579 total

Quite a big improvement only removing excess dependencies, it seems:P

Conclusion: the issue is the cmake dependency extractor; while arguably
the boost_root/ files could be taken as part of the project, files from
/usr/include/wx-3.0 should be definitely 'system' include files (gcc has
some rules about -M and -MM behaviour, I don't know what kind of
extractor cmake uses)

That said... do you really care when compiling just *one* c++ source
usually takes more than all the dependency checking for the whole
project?

-- 
Lorenzo Marcantonio
Logos Srl