Can you prioritize makefile targets?

Question

I have a very large and convoluted make system that takes hours to complete. I'm researching potential optimizations, and I had a question on whether it is possible to prioritize certain targets.

Here's a sample of a simple Makefile that illustrates the concept:

Test%: Prog%
    #run some post build verification on Prog{n}
    #  -- takes 30min or less on failure...

Prog1:  A B
    # do some linking

Prog2:  B C
    # do some linking

A B C:
    # take 10 minutes each

And then I run

make -j2 Prog1 Test2

Then the fastest way to build this would be: B & C, followed by Prog2 & A, followed by Test2 and Prog2. Of course, typically, make would build A and B first, which would delay my final output by 10 minutes.

If I have foreknowledge that Test2 is going to be a bottleneck, is it possible to prioritize that target and all its dependencies over other targets in gnu make?

To add to the question, Let's say I'm building Test1, Test2 ... Test20. Then in that case, I'd like one of the targets -- say Test1 to be completed as soon as possible, as if it that fails, I would like to know as soon as possible so I can terminate the build and let someone else use the build machine.

I considered doing separate instances of make as so:

make Test2 && make Prog1

But, while the test is building, then Prog1 will not be building, resulting in wasted cycles on the build server. If I tried to build them in parrallel with priorities:

`make Test2; nice -n -10 make Prog1`

This could result in race conditions when building B.

I've not yet found any good solutions, but I thought I'd ask on in-case I missed something. (Also, I'm not sure if this should be on SO or SuperUser -- I chose here as it's technically about 'programming', but please feel free to correct me and I can move it).

Answer 1

GNU make offers no syntax to add a weight to a target. I.e. when make is able to start the next job and the independent(!) targets A and B are unblocked (all their dependencies have been fulfilled) and require remaking, then it depends on their order in the internal make database which one gets selected first for execution.

But you could use additional dependencies to achieve your goal. To take the example from your question:

TESTS := Test1 Test2

$(TESTS): Test%: Prog%
Prog1:  A B
Prog2:  B C

ifdef _PRIORITIZE_TEST2
# additional dependencies to make sure Prog2 -> Test2 is prioritized
A: B C
endif

A B C Prog1 Prog2 Test1 Test2 all:
    @echo $@

.PHONY: A B C Prog1 Prog2 $(TESTS) all

Test runs:

$ make --trace -j10 Prog1 Test2
Makefile:13: target 'A' does not exist
echo A
Makefile:13: target 'B' does not exist
echo B
Makefile:13: target 'C' does not exist
echo C
A
B
C
Makefile:13: update target 'Prog1' due to: A B
echo Prog1
Makefile:13: update target 'Prog2' due to: B C
echo Prog2
Prog1
Prog2
Makefile:13: update target 'Test2' due to: Prog2
echo Test2
Test2

$ make --trace -j10 _PRIORITIZE_TEST2=1 Prog1 Test2
Makefile:13: target 'B' does not exist
echo B
Makefile:13: target 'C' does not exist
echo C
B
C
Makefile:13: update target 'A' due to: B C
echo A
Makefile:13: update target 'Prog2' due to: B C
echo Prog2
A
Makefile:13: update target 'Prog1' due to: A B
echo Prog1
Prog2
Makefile:13: update target 'Test2' due to: Prog2
echo Test2
Prog1
Test2

This will probably involve a lot of hand-crafting to get the result you want. You might even need to write different sets of additional dependencies to cover different make invocation scenarios.

You could look into dumping the GNU make database (make --no-builtin-rules --print-data-base), parsing it to extract all targets & their dependencies and visualizing the resulting graph. Here is an example how to generate a directed graph in the DOT language of Graphviz:

#!/usr/bin/perl
use warnings;
use strict;

BEGIN {
    print "digraph build\n";
    print "{\n";
}

my $default_goal = '???';
my @goals;

while (<STDIN>) {
    if (my($target, $dependencies) = /^([\w_-]+):\s+(.*)/) {
        #print "Node ${target}\n";
        print "  $_ -> ${target}\n"
            for (split(' ', $dependencies));

    } elsif (my($goals) = /^MAKECMDGOALS :=\s+(.+)/) {
        @goals = split(' ', $goals);

    } elsif (my($goal) = /^\.DEFAULT_GOAL :=\s+(.+)/) {
        $default_goal = $goal;
    }
}

END {
    @goals = ( $default_goal )
        unless (@goals);
    #print "Root $_\n"
    #   for (@goals);
    print "}\n";
}

exit 0;

For the above example that would result in:

$ make -n --no-builtin-rules --print-data-base Prog1 Test2 2>&1 | perl dummy.pl
digraph build
{
  A -> Prog1
  B -> Prog1
  Prog1 -> Test1
  B -> Prog2
  C -> Prog2
  Prog2 -> Test2
}