mutually-exclusive job scheduling in GNU make?

Question

Using GNU make, is it possible to create a set of targets that will never be scheduled at the same time when using the "--jobs" option?

Background:

To make this a little more concrete, consider a makefile of the form

p1: ...deps... # no parallelization conflicts (can run at the same time as p*, e*)
        ...rules...

p2: ...deps... # no parallelization conflicts (can run at the same time as p*, e*)
        ...rules...

p3: ...deps... # no parallelization conflicts (can run at the same time as p*, e*)
        ...rules...

e1: ...deps... # cannot run at same time as any other e*
        ...rules...

e2: ...deps... # cannot run at same time as any other e*
        ...rules...

e3: ...deps... # cannot run at same time as any other e*
        ...rules...

The main thing I need to do is make sure that e1, e2, and e3 never are being processed at the same time because they do some work on an embedded device with limited resources. They crash if multiple ones of them are executing at the same time. p1, p2, and p3 can be executed in parallel with anything, including any e* job.

Note that the actual makefile has a few thousand targets with a dependency tree that's about 10 levels deep, so I'm hoping there's a way to do this that (a) doesn't require running make serially and (b) preserves the benefits of encoding the dependency tree in a makefile.

Answer 1

One option for you is to use "flock" to run the "e" rules under an exclusive lock. See man flock(1) for details. For example, instead of

e2: deps
    my_cmd foo bar

You can have

e2: deps
    flock .embedded-device-lock -c my_cmd foo bar

What happens then is that all the "e" targets get started by make in parallel (possibly), but the actual commands will be executed serially.

Answer 2

It's not a perfect solution, but you could use an order-only prerequisite to impose a specific ordering on the e* targets:

e1: ...deps...
    ...commands...
e2: ...deps... | e1
    ...commands...
e3: ...deps... | e2 e1
    ...commands...

The prerequisites after the pipe symbol '|' are order-only: they don't force, say, e3 to be updated if e1 or e2 has changed, but they do require that all commands for e1 and e2 finish running before the commands for e3 are started.

The disadvantage of this is that it imposes a specific ordering for these mutually exclusive prerequisites, rather than letting make pick the order, but in practice you can probably figure out a reasonable order manually.