Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parallel makefile requires dependency ordering

I have the following piece of makefile:

CXXFLAGS = -std=c++0x -Wall SRCS     = test1.cpp test2.cpp OBJDIR   = object OBJS     = $(SRCS:%.cpp=$(OBJDIR)/%.o)  all: test1  release: clean test1  test1: $(OBJS)     $(CXX) -o $@ $(OBJS)  $(OBJDIR)/%.o: %.cpp     $(CXX) $(CXXFLAGS) -MD -c -o $@ $<  -include $(SRCS:.cpp=.d)  clean:     rm -rf $(OBJDIR)/*  .PHONY: all clean release  

Now if I try to invoke "make -j4 release" the clean target often gets execute in the middle of building files which causes compilation to fail. My question is how to ensure that the clean target has completed before starting the release build.

like image 306
kyku Avatar asked Dec 13 '11 21:12

kyku


People also ask

How do I run a Makefile in parallel?

To start GNU Make in parallel mode it's enough to specify either the -j or --jobs option on the command-line. The argument to the option is the maximum number of processes that GNU Make will run in parallel. For example, typing make --jobs=4 will allow GNU Make to run up to four subprocesses in parallel.

What is dependency in Makefile?

A dependency is a file that is used as input to create the target. A target often depends on several files. A command is an action that make carries out. A rule may have more than one command, each on its own line.

What is the significance of the order of targets in the Makefile?

The order of rules is not significant, except for determining the default goal: the target for make to consider, if you do not otherwise specify one. The default goal is the target of the first rule in the first makefile. If the first rule has multiple targets, only the first target is taken as the default.


2 Answers

My preference is for

release:     $(MAKE) clean     $(MAKE) test1 

This forces the two targets to be made consecutively without disturbing their inner parallelism (if any).

like image 124
Neil Avatar answered Sep 18 '22 00:09

Neil


You may split the execution into non-parallel (for release) and parallel (for the rest targets) phases.

ifneq ($(filter release,$(MAKECMDGOALS)),) .NOTPARALLEL: endif  release: clean     $(MAKE) test1 

.NOTPARALLEL target will suppress parallel execution if release target is mentioned in the command line. The release target itself will rerun Make after cleaning and build test1 in parallel.

UPD.

More clever solution would also reinvoke Make for each single target in case if there are more than one targets given on the command-line, so that a presence of release target would not force the rest to execute non-parallel too.

ifneq ($(words $(MAKECMDGOALS)),1) .NOTPARALLEL: $(sort all $(MAKECMDGOALS)):     @$(MAKE) -f $(firstword $(MAKEFILE_LIST)) $@ else # ... endif 

Update by James Johnston

The clever solution above doesn't work on versions of GNU make that don't support job servers. For example, released MinGW/native builds of GNU make prior to version 4.0 do not support job servers. (Cygwin/MSYS builds of GNU make do.) The code below uses the .FEATURES variable introduced in make 3.81 to detect if job servers are supported. The symptom of not using this workaround when it's needed is that your "parallel" build will be serialized.

# Check if job server supported: ifeq ($(filter jobserver, $(.FEATURES)),) # Job server not supported: sub-makes will only start one job unless # you specify a higher number here.  Here we use a MS Windows environment # variable specifying number of processors. JOBSARG := -j $(NUMBER_OF_PROCESSORS) else # Job server is supported; let GNU Make work as normal. JOBSARG := endif  # .FEATURES only works in GNU Make 3.81+. # If GNU make is older, assume job server support. ifneq ($(firstword $(sort 3.81 $(MAKE_VERSION))),3.81) # If you are using GNU Make < 3.81 that does not support job servers, you # might want to specify -jN parameter here instead. JOBSARG := endif  ifneq ($(words $(MAKECMDGOALS)),1) .NOTPARALLEL: # The "all" target is required in the list, # in case user invokes make with no targets. $(sort all $(MAKECMDGOALS)):     @$(MAKE) $(JOBSARG) -f $(firstword $(MAKEFILE_LIST)) $@ else  # Put remainder of your makefile here.  endif 
like image 21
Eldar Abusalimov Avatar answered Sep 18 '22 00:09

Eldar Abusalimov