CMake: Redundant linking when modifying shared libraries

Question

I work on a project comprised of a couple dozen shared libraries, each of which has many associated unit tests. Many libs are also dependent on other libs because a lib for some specific functionality will use the code from one of the more common libs. And finally of course there are the production executables which depend on the libs.

There is no question that a change in the API (a header file) of some core common lib should trigger a major recompilation of nearly the entire system. But often there is only a change in the implementation, and the only file compiled is the modified .cxx, and in theory just the modified lib would need to be linked - thanks to dynamic linking there should be no need to relink anything else. But CMake goes ahead and does it anyway: after relinking the lib it relinks all the unit tests associated with that lib. Then it relinks all the libs in that lib's dependency tree and all their unit tests. Finally it relinks the production executables. Due to the scale of the project this takes a lot of precious time.

I have reproduced this behavior with a simple project based on this minimal example (comments removed for brevity and lib changed to shared). My system is Ubuntu 16 on an Intel PC and my CMake version is 3.5.1.

Start with an empty directory and create these files:

CMakeLists.txt

cmake_minimum_required (VERSION 2.8.11)
project (HELLO)
add_subdirectory (Hello)
add_subdirectory (Demo)

Demo/CMakeLists.txt

add_executable (helloDemo demo.cxx)
target_link_libraries (helloDemo LINK_PUBLIC Hello)

Demo/demo.cxx

#include "hello.h"
int main() { hello(); }

Hello/CMakeLists.txt

add_library (Hello SHARED hello.cxx)
target_include_directories (Hello PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

Hello/hello.h

void hello();

Hello/hello.cxx

#include <stdio.h>
void hello() { printf("hello!\n"); }

now run the commands:

mkdir build
cd build
cmake ../
make

You may now execute Demo/helloDemo and see hello!.

Now, touch Hello/hello.cxx and make again. You will see that the helloDemo executable is relinked ("Linking CXX executable helloDemo"). Even if hello.cxx is modified to print a different string the relinked executable remains binary identical so really the relinking was unnecessary.

Is there a way to prevent these redundant build actions?

Answer 1

It turns out that the answer lies in the LINK_DEPENDS_NO_SHARED property. In my example, all that is needed is to add the following line to the Demo/CMakeLists.txt file:

set_target_properties(helloDemo PROPERTIES LINK_DEPENDS_NO_SHARED true)

This will prevent helloDemo from being relinked when one of its dependencies updates - if that dependency is a shared library.

On a more complex system, where some libs are also dependent on other libs, it can be useful to add this setting to their configuration as well.

Thanks to Craig Scott of the CMake mailing list for his help, archived at this link.

Answer 2

Summary

• No proper solution below.
• Patching CMake results in a working solution; but the changes very likely introduce bugs.
• Bazel does not have the same issue (tested), and would likely be significantly faster for your particular use-case.

A Journey

Using the Ninja generator, the resulting build.ninja file (run cmake -G Ninja ..), has the following section. This section clearly shows what is wrong: CMake adds an implicit dependency on Hello/libHello.dylib but an Order-Only-Dependency would be enough.

Complete section comes next, but read below for an explanation, and please scroll to the right:

#############################################
# Link the executable Demo/helloDemo

build Demo/helloDemo: CXX_EXECUTABLE_LINKER__helloDemo Demo/CMakeFiles/helloDemo.dir/demo.cxx.o | Hello/libHello.dylib || Hello/libHello.dylib
  LINK_LIBRARIES = -Wl,-rpath,/Users/myuser/devel/misc/stackoverflow/q50084885/ninja/Hello     Hello/libHello.dylib
  OBJECT_DIR = Demo/CMakeFiles/helloDemo.dir
  POST_BUILD = :
  PRE_LINK = :
  TARGET_FILE = Demo/helloDemo
  TARGET_PDB = helloDemo.dbg

I'm on macOS, for Linux read all *.dylib as *.so.

Note the first non-commented line: build Demo/helloDemo: .... The Ninja-grammar is as follows: build <output>: <rule> <input> | <implicit input> || <order-only-pre-requisite>

<rule> is CXX_EXECUTABLE_LINKER_helloDemo, and Hello/libHelly.dylib is both implicit input as well as order-only prerequisite.

Manually editing the generated build.ninja and removing the implicit input, but not the order-only prerequisite, fixes the problem!

Patching CMake

Patching v3.11.1 with the following patch works (for this specific example). However, it done without deep knowledge of the CMake source code and unit-tests fail. (One of the failing tests is BuildDepends and only fails with the patch not without!)

diff --git a/Source/cmNinjaTargetGenerator.cxx b/Source/cmNinjaTargetGenerator.cxx
index f4faf47a2..bdbf6b948 100644
--- a/Source/cmNinjaTargetGenerator.cxx
+++ b/Source/cmNinjaTargetGenerator.cxx
@@ -239,7 +239,8 @@ cmNinjaDeps cmNinjaTargetGenerator::ComputeLinkDeps() const
 {
   // Static libraries never depend on other targets for linking.
   if (this->GeneratorTarget->GetType() == cmStateEnums::STATIC_LIBRARY ||
-      this->GeneratorTarget->GetType() == cmStateEnums::OBJECT_LIBRARY) {
+      this->GeneratorTarget->GetType() == cmStateEnums::OBJECT_LIBRARY ||
+      this->GeneratorTarget->GetType() == cmStateEnums::EXECUTABLE ) {
     return cmNinjaDeps();
   }

This patch results in generated code, with exactly the same changes as I said can be done manually.

So, this seems to work.

A Trail of Further References

Trying to get rid of build-order dependency

Here, the problem is similar to ours: When compiling target X, with object O and library-dependency L, there is no need to wait for L to be built before compiling object O.

• https://cmake.org/Bug/view.php?id=14726#c35023

• https://cmake.org/Bug/view.php?id=13799

  target_link_libraries adds transitive link dependencies and build order dependencies between the target and its dependencies. In many (if not most) cases the build order dependency is not necessary and results in an overlay constrained dependency graph, which limits parallel execution of the build.

  [Brad King]: FYI, the reason we have these dependencies by default is because the build rules for a library may have custom commands to generate headers or sources that are then used by a target that links to it. This applies even to non-linking targets like static libraries. Furthermore there is no separation of target-level ordering dependencies between compilation and link steps within a single target. Therefore any target that links (shared libs and exes) must have an order dependency on its link dependencies.

  Of course if a project author wants to take responsibility I see no reason not to have an option to skip such dependencies, at least for static libraries. One approach is to add a target property to override the target-level ordering dependencies. This way one could make a static library depend either on nothing or on a subset of its implementation dependencies:

• https://cmake.org/pipermail/cmake-developers/2014-June/010708.html

  My point is that there is no reason to wait building b.cc.o and prog.cc.o; they can be built at the same time as a.cc.o .

  Hence I wonder why libA.so is added as a order-only dependency to b.cc.o when CMake processes this?

Further References

• https://gitlab.kitware.com/cmake/cmake/issues/17666

• https://cmake.org/pipermail/cmake-developers/2016-March/028012.html Maybe a relevant patch, but from March 2016.