Separate compilation of interdependent C modules

Question

Suppose we have a collection of interdependent C modules and we want to create a GNU Makefile for separate compilation of them for a few different builds (e.g., unit tests, user tools, multiple editions).

Each module, while being essential for the complete application, is intended to be used separately or in any reasonable combination with other modules ‒ always exposing the most featured API resulting from the availability of components provided by other modules selected for a particular build.

For the sake of a minimal and complete example, let's assume that our program has three modules (red, green and blue) with all possible conditional functionalities toggled by means of conditional compilation. Each module has two such conditional blocks, each of which enabled by the presence of one of two possible neighbours. This gives us three possible single builds (Red, Green, Blue), three double builds (Cyan, Magenta, Yellow) and one triple build (White) ‒ each containing a dedicated main program (Core) built on top of the set of profided features.

Desired situation

Figure 1 shows the three modules (mod_red.c, mod_green.c and mod_blue.c «RGB»); three areas of cross-module functionality (cyan, magenta and yellow «CMY») implemented within the adjacent modules; and three cores (white, with physical dependencies «RGB» on big, sharpened tops and logical dependencies «CMY» on small tops). Each direction (out of six) denotes a functionality aspect, so the CMY tops pointing out of the main triangle suggest that the synergy may provide additional features.

The desired Makefile is expected to provide recipes for all possible builds, thus use four versions of each of the three modules and seven different cores. It should also be smart enough to avoid brutal solution (full block of gcc commands for each recipe) and to keep the advantages of separate compilation.

Without separate compilation the problem is easy (at least for unilateral dependencies): main program includes necessary sources and the dependent blocks are enabled by preprocessor flags, e.g. those set by other modules' include guards. With separate compilation, however, the set of modules comprising a particular build is unknown to the compiler.

Manual approach

The desired situation could be achieved manually with the shell commands listed below.

# Single objects:
gcc -c -o mod_green.o mod_green.c

# Double objects
gcc -c -include mod_blue.h -o mod_red+B.o mod_red.c
gcc -c -include mod_red.h -o mod_blue+R.o mod_blue.c

# Triple objects
gcc -c -include mod_green.h -include mod_blue.h -o mod_red+G+B.o mod_red.c
gcc -c -include mod_red.h -include mod_blue.h -o mod_green+R+B.o mod_green.c
gcc -c -include mod_red.h -include mod_green.h -o mod_blue+R+G.o mod_blue.c

# Builds
gcc -o green green.c mod_green.o
gcc -o magenta magenta.c mod_red+B.o mod_blue+R.o
gcc -o white white.c mod_red+G+B.o mod_green+R+B.o mod_blue+R+G.o

As for the desired situation, this example shows only the three representative builds: Green, Magenta and White. Others are formed analogously.

Classic approach

With a classic Makefile solution the Green build stays the same, but the other two have missing logical dependencies (i.e., the CMY-provided symbols). It is so because the building process is currently (and usually) defined as follows:

white: white.c mod_red.o mod_green.o mod_blue.o
    gcc -o $@ $^

magenta: magenta.c mod_blue.o mod_red.o 
    gcc -o $@ $^

green: green.c mod_green.o
    gcc -o $@ $^

%.o: %.c
    gcc -c -o $@ $<

Here the problem is clearly exposed: the last rule does not distinguish between particular builds ‒ the context is lost. Also, I need to end up with different binary versions of each module to satisfy different builds. What is the proper way to do it?

Answer 1

This should work. Didn't try it though. I think it'll be easy to create the other rules from this point.

BINARY = build

CC = gcc

SOURCES_RGB = rgb.c mod_red.c mod_green.c mod_blue.c
OBJECTS_RGB = $(SOURCES_RGB:.c=_rgb.o)

BINARY_RGB = $(addprefix RGB-,$(BINARY))

CFLAGS_RGB = -include mod_rgb.h

$(BINARY_RGB): $(OBJECTS_RGB)
        $(CC) -o $@ $^

%_rgb.o: %.c
        $(CC) -c $(CFLAGS_RGB) -o $@ $<

Answer 2

With GNU Make 3.82 it is possible to define canned recipes which – after being made parametric – might be used as templates for the double/triple objects' build specifications. The templates can be then instantiated with Make's unique $(call var,par,...) function and evaluated with the very special $(eval code), available since version 3.8. This trick allows us to avoid a lot of boilerplate code and enables Makefile to automatically adapt to forthcoming project growth.

While having all possible module-building rules available, we are one step away from the goal; this is a quite tricky step however. We need to rebuild each build's prerequisites to explicitly expose each inter-module dependency. It is done by replacing the single module objects with the special cases arising from the ambient neighbourhood (e.g., a superficial mod_red.o mod_green.o is substituted with an explicit mod_red+G.o mod_green+R.o). The substitution is handled by the discover macro anclosing each prerequisite list and driven by the xdepend global variable, which specifies the cross dependencies – in practice, only a few modules will depend on one another.

# Cross dependency specification (dependent+dependency)
xdepend := blue+red red+blue

# Test for string equality and set membership
equals = $(if $(subst $1,,$2),$(empty),true)
exists = $(if $(filter $1,$2),true,$(empty))

# Extract of the first and second member of a dependency token
pred = $(firstword $(subst +, ,$1))
succ = $(lastword $(subst +, ,$1))

# Rebuild prerequisites to expose modules' interdependencies
discover = $(strip $(foreach mod,$(basename $1),\
    $(subst $(space),,$(obj)/$(mod)\
    $(foreach dep,$(xdepend),\
        $(and\
            $(call equals,$(call pred,$(dep)),$(mod)),\
            $(call exists,$(call succ,$(dep)),$(basename $1)),\
            $(lnk)$(call succ,$(dep))\
        )\
    ).o)\
))


# Create compilation rules for interdependent module objects
define rule
$(obj)/$1$(lnk)$2.o: $(src)/$1.c $(inc)/$1.h $(inc)/$2.h | $(obj)
    $(CC) $(CFLAGS) -include $(inc)/$2.h -c $(src)/$1.c -o $(obj)/$1$(lnk)$2.o

endef

$(foreach dep,$(xdepend),\
    $(eval $(call rule,$(call pred,$(dep)),$(call succ,$(dep))))\
)

Armed with the above definitions, we can now build our project in the following way:

# Rules for Magenta Build and intermediate objects
magenta: $(call discover, magenta.o mod_red.o mod_blue.o)
    $(CC) $(CFLAGS) $^ -o $@ $(LDFLAGS)

$(obj)/%.o: $(src)/%.c $(inc)/%.h | $(obj)
    $(CC) $(CFLAGS) -c -o $@ $<

$(obj):
    mkdir $(obj)

For further clarification and the most recent knowledge – read GNU Make Manual.