cmake force parallel .C.o compilation before linking

Question

I have a project with 5 libraries that each comes from 10 C++ source files, then 10 executables that depend on the libraries. Each library also depends on the previous one. Using CMake and then 'make -j 50' on a 48-core Ubuntu machine, I first have to wait for each library to build on 10 cores or fewer (mostly fewer, as one .C file takes several minutes to compile), then my executables build in parallel.

I would like to first run all the .C.o compilations in parallel on all .C source files (I know how to get that list into a CMake variable), then running only the linker in the order specified by the dependencies.

Is there a way to do that with CMake, for example by setting a phony target or something similar? (I only want to recompile the .C files that have been modified).

Answer 1

A way to do this is to add a false target consisting of all the sources for each of the library targets in CMake. An example CMakeLists.txt might be...

set(ONE_SRC one/a.c one/b.c)
set(TWO_SRC two/a.c two/b.c)

# pre-compile
add_library(all ${ONE_SRC} ${TWO_SRC})

add_library(one ${ONE_SRC})
add_library(two ${TWO_SRC})

add_dependencies(one all)
add_dependencies(two one)

The liball.so target will compile all the sources necessary for all the other libs, and it should break the false dependency between the source files of libone.so and libtwo.so when you run make, ninja, whatever... it will maximize compile parallelism across targets.

CMake may not be smart enough to compile ${ONE_SRC} ${TWO_SRC} files only once. However, this can be fixed using ccache, which will cache pre-processed files. It has the added benefit of reducing any identical re-compile times.

A simple config for ccache is to add symlinks to the ccache binaries in a local $PATH.

jason@io ~ $ ll ~/bin/ccache
total 0
lrwxrwxrwx 1 jason jason 15 May 12  2013 c++ -> /usr/bin/ccache*
lrwxrwxrwx 1 jason jason 15 May 12  2013 cc -> /usr/bin/ccache*
lrwxrwxrwx 1 jason jason 15 Apr 27 21:38 clang -> /usr/bin/ccache*
lrwxrwxrwx 1 jason jason 15 Apr 27 21:38 clang++ -> /usr/bin/ccache*
lrwxrwxrwx 1 jason jason 15 May 12  2013 g++ -> /usr/bin/ccache*
lrwxrwxrwx 1 jason jason 15 Oct  6  2013 gcc -> /usr/bin/ccache*
lrwxrwxrwx 1 jason jason 15 May 12  2013 x86_64-pc-linux-gnu-c++ -> /usr/bin/ccache*
lrwxrwxrwx 1 jason jason 15 May 12  2013 x86_64-pc-linux-gnu-g++ -> /usr/bin/ccache*
lrwxrwxrwx 1 jason jason 15 May 12  2013 x86_64-pc-linux-gnu-gcc -> /usr/bin/ccache*

The ccache environment variables are relatively straightforward.

# ccache

export CCACHE_DIR="/var/ccache/${USER}"
export CCACHE_SIZE="4G"
export CCACHE_COMPRESS="1"