Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

GNU make wildcard function doesn't find runtime generated files

Summary:
I'm using GNU Make (3.81) on a unix-like system and I've run into an issue where the $(wildcard, pattern) function is unable to find a file generated by a (presumably/apparently?) previously executed recipe, whereas other programs (e.g. ls) are able to verify its existence. I would like to know why the wildcard function is not returning anything, when it's expanded (to the empty string), and how I might get it to find the generated file.

Test case:
The following test case illustrates the problem.
Makefile contents:

.PHONY: build clean

test:
    @echo "Creating test file 'test'."
    @echo "this is a test file" > test

build: test
    @echo "Directory contents:"
    @ls
    @echo "Test file contents:"
    @cat test
    @echo "Wildcard output:"
    @echo $(wildcard test)

clean:
    @rm -f test

Running the makefile twice (then cleaning up) shows that only on the second run does it detect the created file.
Output:

Creating test file 'test'.
Directory contents:
makefile  test
Test file contents:
this is a test file
Wildcard output:

Directory contents:
makefile  test
Test file contents:
this is a test file
Wildcard output:
test

To reproduce:
Save the makefile in an empty directory, then run "make build;make build;make clean".

Research:
The order in which the prerequisites of a target are processed is independent of the order in which they appear in the list after the colon, therefore explicit inter-prerequisite dependencies must be introduced if you want to define the order in which they're processed.

However, I don't think I can strip down the dependencies any more; build depends on test, and test corresponds to a file which doesn't exist and is generated by its recipe. I presumed it would be present by the time the recipe for build was to be processed.

From my knowledge of how make parses the makefile, the wildcard function inside the recipe should not be expanded until it's processed. However, I clearly see a difference between the output of the built-in make functionality, and the output of common utilities accessed through the shell, so this presumption is probably wrong, but I haven't come across any good explanations of what's supposed to happen here, not even in the GNU make manual.

N.B. The purpose of this question is simply to attain an understanding of the workings of the wildcard function relevant to this problem; there is a practical use case from which I distilled this watered down version, but I'm more interested in this case.

like image 974
Francis Avatar asked Feb 24 '13 02:02

Francis


1 Answers

GNU make expands all of the commands for a target before executing any of them. In this context, "expand" means "replace all variable references with their values (recursively), and evaluate any function references." $(wildcard) is a function reference, of course. Since the expansion occurs before any of the commands are executed, of course the file test cannot be found -- it hasn't been created yet. The second time you run the build, test exists already, so $(wildcard) can find it.

Regarding your statement that the order in which prerequisites are processed: technically GNU make makes no guarantees about the order, but practically speaking the prereqs are processed left-to-right, and despite attempts by purists to change that (for the sole purpose of screwing with people who haven't completely specified dependencies), the likelihood that GNU make will actually change that implementation detail is vanishingly small. However you can only really rely on that ordering in a serial build. If you run the build in parallel, then things get a bit trickier: GNU make does still process the prereqs in left-to-right order, but it no longer waits for one to finish before starting the next, so in effect they may appear to be processed in random or "any" order, subject to the inter-prereq dependencies, which is why it's a good practice to specify the those dependencies.

like image 97
Eric Melski Avatar answered Oct 22 '22 01:10

Eric Melski