Gnu makefile looping through list, first item is empty and last item is skipped

I have a makefile with a loop that should iterate through the elements in a list and pass each element to a function. In the first iteration, it seems to pass a blank element, and then it never passes the last element.

Here's a makefile that exhibits the problem:

ALL_BENCHMARKS=bench_1 bench_2 bench_3 bench_4

define GENERATE_TARGET
BENCHMARK=$(1)
COMPILER=$(2)
./$(BENCHMARK)$(COMPILER).file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))
endef

$(foreach benchmark, $(ALL_BENCHMARKS), \
    $(info loop: $(benchmark)) \
    $(eval $(call GENERATE_TARGET,$(benchmark),.llvm)) \
)

default:
    @echo "default"

I saved it as loop.mak and run it like this:

make -f loop.mak

It gives me this output:

$ make -f loop.mak
loop: bench_1
---- BENCHMARK:  COMPILER:
loop: bench_2
---- BENCHMARK: bench_1 COMPILER: .llvm
loop: bench_3
---- BENCHMARK: bench_2 COMPILER: .llvm
loop: bench_4
---- BENCHMARK: bench_3 COMPILER: .llvm
make: 'bench_1.llvm.file' is up to date.

Note the blanks in the output the first time, and it never prints ---- BENCHMARK: bench_4.

I'm running it with this make on Ubuntu 22.04:

$ make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I tried running the makefile as written above and expected it to print all 4 elements of ALL_BENCHMARKS. Instead, it printed a blank the first time and then the first three elements in ALL_BENCHMARKS.

Solution

The main issue is the double expansion of the eval parameter. When using foreach-eval-call to programmatically instantiate make constructs it is important to remember that the eval parameter is expanded twice: a first time when calling eval, and a second time when make parses the instantiated constructs. With your code, during the first call of eval, what happens is the following:

eval expands its parameter, starting with call that substitutes $(1) and $(2), leaving:
```
BENCHMARK=bench_1
COMPILER=.llvm
./$(BENCHMARK)$(COMPILER).file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))
```
Then, eval continues the expansion and because BENCHMARK and COMPILER are not yet defined the result is:
```
BENCHMARK=bench_1
COMPILER=.llvm
./.file:
```
At this step the info macro is also expanded and prints ---- BENCHMARK: COMPILER: . info expands as the empty string, so the rule for target ./.file has no recipe.
make parses this and expands what must be immediately expanded (see the manual for an explanation about immediate vs. deferred), that is, nothing. The instantiated constructs are thus exactly:
```
BENCHMARK=bench_1
COMPILER=.llvm
./.file:
```

You can verify this by trying make -f loop.mak ./.file. As you will see make does not complain; the rule is defined.

During the second call to eval things are different because 2 (recursive) variable definitions have been instantiated for BENCHMARK (bench_1) and COMPILER (.llvm). The resulting instantiated make constructs are thus:

BENCHMARK=bench_2
COMPILER=.llvm
./bench_1.llvm.file:

The info macro prints ---- BENCHMARK: bench_1 COMPILER: .llvm. I think you got the idea. And of course the rule with bench_4 is never instantiated.

To solve this issue you must escape the undesired expansions by doubling the $ signs (and sometimes even more than doubling). See the manual for the full explanation. In your case you can try:

$ cat loop.mak
ALL_BENCHMARKS=bench_1 bench_2 bench_3 bench_4

define GENERATE_TARGET
BENCHMARK=$(1)
COMPILER=$(2)
./$$(BENCHMARK)$$(COMPILER).file:
    $$(info ---- BENCHMARK: $$(BENCHMARK) COMPILER: $$(COMPILER))
endef

$(foreach benchmark, $(ALL_BENCHMARKS), \
    $(info loop: $(benchmark)) \
    $(eval $(call GENERATE_TARGET,$(benchmark),.llvm)) \
)

default:
    @echo "default"

$ make -f loop.mak ./bench_{1..4}.llvm.file
loop: bench_1
loop: bench_2
loop: bench_3
loop: bench_4
---- BENCHMARK: bench_4 COMPILER: .llvm
make: 'bench_1.llvm.file' is up to date.
---- BENCHMARK: bench_4 COMPILER: .llvm
make: 'bench_2.llvm.file' is up to date.
---- BENCHMARK: bench_4 COMPILER: .llvm
make: 'bench_3.llvm.file' is up to date.
---- BENCHMARK: bench_4 COMPILER: .llvm
make: 'bench_4.llvm.file' is up to date.

Note that the recipes now use $$(info ...). The first expansion by eval leaves $(info ...). The second expansion happens only just before make passes the recipe to the shell. As, at that time, BENCHMARK expands as bench_4, info prints the same message for the 4 rules. This also explains why the info messages are now interleaved with the make: 'bench_N.llvm.file' is up to date. messages while they were initially interleaved with the loop: bench_N messages: the expansion of the info macros has been deferred from the parsing phase to the execution phase.

To better understand how this new version works you can imagine the result after the four eval expansions, and before the parsing by make:

ALL_BENCHMARKS=bench_1 bench_2 bench_3 bench_4

BENCHMARK=bench_1
COMPILER=.llvm
./$(BENCHMARK)$(COMPILER).file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))

BENCHMARK=bench_2
COMPILER=.llvm
./$(BENCHMARK)$(COMPILER).file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))

BENCHMARK=bench_3
COMPILER=.llvm
./$(BENCHMARK)$(COMPILER).file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))

BENCHMARK=bench_4
COMPILER=.llvm
./$(BENCHMARK)$(COMPILER).file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))

default:
    @echo "default"

Next, make parses all this and expands what needs to be (in this case only the targets of the rules), leaving the recipes unmodified. Each time it needs the value of a variable it uses the most recently assigned value:

ALL_BENCHMARKS=bench_1 bench_2 bench_3 bench_4

BENCHMARK=bench_1
COMPILER=.llvm
./bench_1.llvm.file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))

BENCHMARK=bench_2
COMPILER=.llvm
./bench_2.llvm.file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))

BENCHMARK=bench_3
COMPILER=.llvm
./bench_3.llvm.file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))

BENCHMARK=bench_4
COMPILER=.llvm
./bench_4.llvm.file:
    $(info ---- BENCHMARK: $(BENCHMARK) COMPILER: $(COMPILER))

default:
    @echo "default"

Then, when executing any of these 4 recipes, make first expands it with the most recently assigned values of BENCHMARK(bench_4) and COMPILER(.llvm)...

But, as suggested in another answer, you don't need intermediate variables. The following would do (almost) the same:

$ cat loop.mak
ALL_BENCHMARKS=bench_1 bench_2 bench_3 bench_4

define GENERATE_TARGET
./$1$2.file:
    $$(info ---- BENCHMARK: $1 COMPILER: $2)
endef

$(foreach benchmark, $(ALL_BENCHMARKS), \
    $(info loop: $(benchmark)) \
    $(eval $(call GENERATE_TARGET,$(benchmark),.llvm)) \
)

default:
    @echo "default"

$ make -f loop.mak ./bench_{1..4}.llvm.file
loop: bench_1
loop: bench_2
loop: bench_3
loop: bench_4
---- BENCHMARK: bench_1 COMPILER: .llvm
make: 'bench_1.llvm.file' is up to date.
---- BENCHMARK: bench_2 COMPILER: .llvm
make: 'bench_2.llvm.file' is up to date.
---- BENCHMARK: bench_3 COMPILER: .llvm
make: 'bench_3.llvm.file' is up to date.
---- BENCHMARK: bench_4 COMPILER: .llvm
make: 'bench_4.llvm.file' is up to date.

Note that this time the 4 recipes have been expanded with different info messages.