Parallelization of recursive jobs in GNU make

Question

I am looking for an elegant way for the parallelization of jobs in GNU make. Here is a sample of what I did so far. Make processes the directories dir-1, dir-2 and dir-3 in a serial fashion which is logical but not my intention:

SUBDIRS=dir-1 dir-2 dir-3

default: all

all:
  @for dir in $(SUBDIRS); do (cd $$dir; $(MAKE)); done

.PHONY: clean

clean:
  @for dir in $(SUBDIRS); do (cd $$dir; $(MAKE) clean); done

Is there a way to support parallel processing of these directories using the "-j" option without specifying specific targets for each directory?

Answer 1

SUBDIRS = a b c

default: all

$(SUBDIRS)::
    $(MAKE) -C $@ $(MAKECMDGOALS)

all clean : $(SUBDIRS)

Answer 2

This probably will not answer your question directly, but besides what the other answers suggest, I would recommend to look into non-recursive make techniques. That is truly an elegant way to parallelize build, although, depending on what are the existing Makefiles, can require significant effort. Non-recursive make has advantages not limited to easy parallelization: it sees a complete dependency graph and so it does not need to build too little or too much, meaning faster (sometimes much faster) build times.

Some resources:

• Discussion on SO
• The classic text about trouble with recursive make
• Design of non-recursive makefile
• Implementation
• Another implementation
• Benchmarks (quite outdated though)