manipulate build order for independent targets in ninja/cmake

Question

I'm working on a c++ project built with cmake+ninja with approx 1200 build targets on a 64 thread computer.

There's one translation unit that takes 10min to compile, most others are comparably fast such that a build of all other targets together takes (building on all threads) only about 9 minutes. The slow translation unit is rather independent of the rest, so it doesn't have to be scheduled late, but as it turns out is is scheduled late by default, such that a complete build takes me between 15 and 20 minutes, and at the end there is only one thread working with almost all other targets done. The build would be faster for me, if the slow translation unit would be scheduled first, blocking one thread for about 10 minutes, while all other threads work on the rest of the project and the entire project is built within 10min.

Is there a way in cmake or ninja to shift the priorities for scheduling to point out slow or "please early" targets without messing up dependencies?

Answer 1

As of February 27, 2021, the answer is no. There are some open issues (#232, #376) and an abandoned PR (#1333) on GitHub requesting this feature in base Ninja. CMake does not provide any way to prioritize a target (at least through 3.20) either.

Messing with dependencies (even order-only) doesn't help here (as you likely know) because that would just force smaller targets to start either completely before or completely after the long target. A priority hint is what's truly needed here.

The only workaround I can think of (and it's not a great one), is to split your long target out into a separate ExternalProject and create a superbuild that builds the large target and the independent portion at the same time. This would require significant restructuring and would be a non-starter for a lot of projects. It might be worth the pain if you're losing a lot of development time to this issue, though.