makefile : How to link object files from different subdirectory and include different search paths

Question

I want to change the location of my test code (tsnnls_test_DKU.c) and I am unable to make change in makefile to reflect this folder change properly. Some help would be appreciated.

I have two questions: 1) How to link object files from different subdirectory 2) include different search paths (3 search paths in my example).

In my orinal setup, where makefile works fine, I put my test code tsnnls_test_DKU.c at following location (inside the third party libraries):

Dir1 = /home/dkumar/libtsnnls-2.3.3/tsnnls

All object files, that I am linking to, are at

OBJDir = /home/dkumar/libtsnnls-2.3.3/tsnnls

Also, some include file contained in tsnnls_test_DKU.c are at following three locations (three search paths):

Dir1 = /home/dkumar/libtsnnls-2.3.3/tsnnls  
Dir2 = /home/dkumar/libtsnnls-2.3.3
Dir3 = /home/dkumar/libtsnnls-2.3.3/tsnnls/taucs_basic

and my makefile works fine.

However, I would like to change the location of test code to:

Dir4 = /home/dkumar/CPP_ExampleCodes_DKU/Using_tsnnls_DKU/

Here is how my makefile looks like (Updated after inputs from other user:

# A sample Makefile

VPATH = -L/home/dkumar/libtsnnls-2.3.3/tsnnls
INC_PATH  = -I/home/dkumar/libtsnnls-2.3.3/ -I/home/dkumar/libtsnnls-2.3.3/tsnnls/  -I/home/dkumar/libtsnnls-2.3.3/tsnnls/taucs_basic/

# Here is a simple Make Macro.
LINK_TARGET     = tsnnls_test_DKU
OBJS_LOC    = tsnnls_test_DKU.o

# Here is a Make Macro that uses the backslash to extend to multiple lines.
OBJS =  libtsnnls_la-taucs_malloc.o libtsnnls_la-taucs_ccs_order.o \
    libtsnnls_la-taucs_ccs_ops.o libtsnnls_la-taucs_vec_base.o \
    libtsnnls_la-taucs_complex.o libtsnnls_la-colamd.o \
    libtsnnls_la-amdbar.o libtsnnls_la-amdexa.o \
    libtsnnls_la-amdtru.o libtsnnls_la-genmmd.o \
    libtsnnls_la-taucs_timer.o libtsnnls_la-taucs_sn_llt.o \
    libtsnnls_la-taucs_ccs_base.o libtsnnls_la-tlsqr.o \
    libtsnnls_la-tsnnls.o libtsnnls_la-lsqr.o   \
    $(OBJS_LOC)

REBUILDABLES = $(LINK_TARGET) 

all : $(LINK_TARGET)
    echo All done

clean : 
    rm -f $(REBUILDABLES)   
    echo Clean done

#Inclusion of all libraries
RANLIB = ranlib
STATICLIB= /usr/local/lib/taucs_full/lib/linux/libtaucs.a 

tsnnls_test_LDADD = $(LDADD)
LIBS = -largtable2 -llapack -lblas -lquadmath -lm

$(LINK_TARGET) : $(OBJS)   $(tsnnls_test_LDADD) $(LIBS)  $(STATICLIB)
gcc -g ${INC_PATH} -o $@ $^

The error that I get when trying to run "$make"

make: *** No rule to make target `libtsnnls_la-taucs_malloc.o', needed by `tsnnls_test_DKU'.  Stop.

Obviously, I have not been able to use VPATH properly.

UPDATE: Thanks to Mike Kinghan for answering my question.

Answer 1

Q1: How to link object files from a different subdirectory?

Let's say your program prog is a C program that will be linked from object file0.o, file1.o which are to be compiled into subdirectory obj. Here is the sort of thing you typically need in your makefile to perform that linkage.

$(OBJS) = $(patsubst %.o,obj/%.o,file0.o file1.o)

prog: $(OBJS)
    gcc -o $@ $(CFLAGS) $(OBJS) $(LDFLAGS) $(LDLIBS)

This makes $(OBJS) = obj/file0.o, obj/file1.o and you simply pass the object files like that to the link command. Documentation of patsubst

N.B. This is not sufficient to create the obj subdirectory if it doesn't exist when you want to compile an object file into it. You'll have to create it yourself or study how to make make do it.

Q2: How to include different search paths?

This is an ambiguous question - the ambiguity is confusing you - and we must break it down into Q2.a, Q2.b, Q2.c:

Q2.a: How to specify different search paths where the preprocessor will look for header files that are #include-ed in the source code?

By default, the preprocessor will look for header files using a built-in a list of standard search paths. You can see them by running the preprocessor in verbose mode, e.g.cpp -v (CTRL-C to terminate). The output will contain something like:

#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-linux-gnu/4.8/include
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

Let's suppose you have some header files of your own in subdirectories inc and inc/foo/bar and want the preprocessor to search these directories as well. Then you need to pass the preprocessor options:

-I inc -I inc/foo/bar

to your compile command. Preprocessor options are conventionally assigned to the make variable CPPFLAGS, e.g.

CPPFLAGS = -I inc -I inc/foo/bar

(along with any other preprocessor options you require), and passed via this variable in the compile command recipe, e.g.

gcc -c -o $@ $(CPPFLAGS) $(CFLAGS) $<

N.B. It is a common mistake to think that CPPFLAGS is the conventional make variable for C++ compiler flags. The conventional make variable for C++ compiler flags is CXXFLAGS.

You can see the effect of the -I option by running:

mkdir -p inc/foo/bar # Just to create the path
cpp -v -I inc -I inc/foo/bar

(CTRL-C to terminate). Now the output will contain the like of:

#include "..." search starts here:
#include <...> search starts here:
 inc
 inc/foo/bar
 /usr/lib/gcc/x86_64-linux-gnu/4.8/include
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

Q2.b: How to specify different search paths where the the linker will look for libraries?

Let's suppose you are have a library, libfoobar.a that you need to link with prog and that it resides in a directory lib that is 2 levels up from your makefile. Then you need to pass the linker options:

-L ../../lib

and

-lfoobar

to your link command. The first of these will tell the linker that ../../lib is one of the places to look for libraries. Conventionally, you pass this option in the linker command recipe via LDFLAGS. The second tells the linker to search for some library called libfoobar.a (a static library) or libfoobar.so (a dynamic library). Conventionally, you pass this option in the linker command recipe via LDLIBS

Just as there is a default list of search paths for the preprocessor, there is a default list of search paths for the linker. You can see them by running:

gcc -v -Wl,--verbose 2>&1 | grep 'LIBRARY_PATH'

The output will be something like:

LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/: /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/: /usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../:/lib/:/usr/lib/

When you need to link one of the standard libraries, e.g. libm (the math library), that resides in one of the the default library search paths, you don't need to pass any -L option. -lm alone will do.

Q2.c: How to specify different search paths where make will look for the prerequisites of targets?

N.B. This is a question about make, not a question about the preprocessor, compiler, or linker.

We have assumed that all of your object files will be compiled into the subdirectory obj. To get them compiled there, it would be nice and simple just to use the pattern rule:

obj/%.o:%.c
    gcc -c -o $@ $(CPPFLAGS) $(CFLAGS) $<

which tell make that, e.g. obj/file0.o is made from file0.c by the recipe:

gcc -c -o obj/file0.o $(CPPFLAGS) $(CFLAGS) file0.c

and similarly for any file obj/*.o and matching file *.c

This is fine as long file0.c resides in the same directory as the makefile, but suppose you have your *.c files somewhere else? Say your source files are organised in subdirectories, foo/file0.c and bar/file1.c. Then make will be unable to satisfy that pattern rule and will say there is "no rule to make target obj/file0.o", etc.

To solve this problem use VPATH, a make variable that has a special meaning. If you assign a list of directory names, punctuated by ':', to VPATH, then make will search for a prerequisite in each of the listed directories whenever it can't find it in the current directory. So:

VPATH = foo:bar

will cause make to look in the current directory, and then foo and bar when it tries to find .c files to match that pattern rule. It will succeed in satisfying the rule and will compile the necessary source files.

N.B. You have used VPATH wrongly in your posted code:

VPATH = -L/home/dkumar/libtsnnls-2.3.3/tsnnls

You have asigned a linker search path to it, with the linker option -L, which has no business there.

Bottom line:

• The preprecessor search paths, for locating header files, are specified with the preprocessor's -I<dirname> option. Pass these options to the compile recipe in CPPFLAGS

• The linker search paths, for locating libraries, are specified with the linker's -L<dirname> option. Pass these options to the linkage recipe in LDFLAGS

• The search paths for the preprequisities of make rules are specified in the make variable VPATH, as a ':-punctuated list of directory names.