← Back to team overview

kicad-developers team mailing list archive

Re: Bug 1833851 - Massively Parallel Builds

 

Hi,

> The background is the GNU make is not intelligent about parallel
> builds and doesn't merge generated targets.

FWIW we have the same problem on MSBuild. The problem we have is that we
use the same generated file from two different projects, and CMake's
paradigm is that generated files belong to the project that uses them, so
project-to-project dependencies are used to lock out parallel generation.

For both Make and MSBuild, the same target inside the same project is
merged and handled correctly. The Makefile generator uses sub-make
invocations for the separate projects, and each of these only sees the
dependency tree for its project, which causes the problem.

> 1) Build the header file into a target directory instead of the
> source directory.  Then everyone gets to build their own with no
> conflict.  We end up building lots and lots of header/lexer files.
> Developers won't see the files in the usual location.

This is a good idea anyway. When building in a separate directory, the
source tree should not be touched. If you keep the source tree in a
subdirectory "src" below a directory containing a Makefile like

    all: build-1 build-2 build-3 build-4
    build-%:
        mkdir -p build/$@
        (cd build/$@ && cmake ../../src)
        $(MAKE) -C build/$@

this should be safe to build in parallel (and might be useful if you want
to build several configurations, because the toplevel make has the
jobserver).

In principle, the original "generated files live in a separate project"
approach should work, but that project needs to be a library which consumes
the generated cpp file (so ownership of the custom rule is clear) and
exports the header file to other projects. We'd have to clarify whether
this is safe with the generated dependencies for headers if the token list
changes, but I think it should be.

I might be able to experiment with that a bit tonight if noone beats me to
it.

   Simon


References