Using GNU Make to compile all .c files in a subdirectory

I'm working on a project in C, and I'm using GNU Make to handle compiling everything, but I'm having a lot of trouble trying to compile every .c file in my project, including ones in subdirectories. Specifically, I wanted to take every .c file in a source directory, compile everything into .o files in a separate out/ directory, and then link everything into an executable file, while also doing dependency checking to avoid recompiling unmodified files. I tried a lot of stuff, most of it failing, and this is my current attempt:

SRC_DIR := src/main
HDR_DIR := src/headers
OUT_DIR := out
BIN_DIR := $(OUT_DIR)/bin
TARGET  := $(BIN_DIR)/example

CC     := gcc
CFLAGS := -Werror -Wall -Wextra -Wpedantic -I $(HDR_DIR)

SOURCES   := $(shell find $(SRC_DIR) -name *.c)
FIND_SRC   = $(shell find $(SRC_DIR) -name $(patsubst %.o,%.c,$(notdir $(1))))
OBJS      := $(patsubst %.c,$(OUT_DIR)/%.o,$(notdir $(SOURCES)))

all: $(TARGET)

$(TARGET): $(OBJS)
        $(CC) $(OBJS) -o $(TARGET) $(CFLAGS)

$(OBJS): $(OUT_DIR)/%.o: $(call FIND_SRC,%)
        $(CC) -c $^ -o $@

clean:
        rm -f $(TARGET)
        rm -f $(OUT_DIR)/*.o

The issue is that the $^ variable is getting set to nothing, so it's not actually compiling anything. Here's the output:

gcc -c  -o out/main.o
gcc: fatal error: no input files
compilation terminated.
make: *** [Makefile:20: out/main.o] Error 1

It's trying to compile nothing, when it should be trying to compile src/main/main.c

My question is this: is there a way to do what I want to do using just GNU Make? If this is possible, then how could I do that? If it isn't, then what other alternatives to GNU Make should I try so I can get the result I wanted?

Thank you!

Solution

The problem is that in a make rule the list of prerequisites is expanded immediately when make parses the makefile while you would like it to be expanded at a later stage (deferred). So your $(call FIND_SRC,%) expands as (step by step):

$(shell find $(SRC_DIR) -name $(patsubst %.o,%.c,$(notdir %)))
$(shell find $(SRC_DIR) -name $(patsubst %.o,%.c,%))
$(shell find $(SRC_DIR) -name %)
probably the empty string because you do not have files named % under src/main. If you had one, things would not be better because the compilation rule would become:
```
$(OBJS): $(OUT_DIR)/%.o: src/main/some/path/%
        echo $(CC) -c $^ -o $@
```
which is not what you want.

Anyway, there is an important issue with your idea of a flat directory of object files: if you have source files src/main/foo.c and src/main/bar/foo.c there will be a name collision on out/foo.o and all this will fail. A safer and simpler solution would be to replicate the source hierarchy in out:

SRC_DIR := src/main
HDR_DIR := src/headers
OUT_DIR := out
BIN_DIR := $(OUT_DIR)/bin
TARGET  := $(BIN_DIR)/example

CC      := gcc
CFLAGS  := -Werror -Wall -Wextra -Wpedantic -I $(HDR_DIR)

SOURCES := $(shell find $(SRC_DIR) -type f -name '*.c')
OBJS    := $(patsubst $(SRC_DIR)/%.c,$(OUT_DIR)/%.o,$(SOURCES))

all: $(TARGET)

$(TARGET): $(OBJS) | $(BIN_DIR)
        $(CC) $^ -o $@

$(OUT_DIR)/%.o: $(SRC_DIR)/%.c
        mkdir -p $(@D) && $(CC) $(CFLAGS) -c $^ -o $@

clean:
        rm -f $(TARGET) $(OBJS)

$(BIN_DIR):
        mkdir -p $@

Notes:

We must create the sub-directories in out before we store object files in them; this is what mkdir -p $(@D) && ... does. $(@D) expands as the directory of the target and the -p option of mkdir also creates the parent directories if needed (and does not raise an error if the directory exists).
We fix your find command to search only for files (-type f) and avoid the expansion of *.c by the shell ('*.c' instead of *.c).
We use automatic variables $@ and $^ instead of $(TARGET) and $(OBJS) in the link rule for easier maintenance.
We use $(OBJS) in the clean rule instead of $(OUT_DIR)/*.o that could delete more than wanted and would not recurse in sub-directories. If the out directory contains only your object files and the executable, as it is now recreated automatically, you could as well clean with rm -rf $(OUT_DIR).
We add $(BIN_DIR) as an order-only prerequisite of the link rule, and a rule to create it automatically if it is missing.
We move $(CFLAGS) from the link recipe where it is useless, to the compilation recipe where it is needed.

But if you are 100% sure that name collisions cannot happen and you absolutely want a flat directory of object files, as you use GNU make, you could use vpath:

SRC_DIR := src/main
HDR_DIR := src/headers
OUT_DIR := out
BIN_DIR := $(OUT_DIR)/bin
TARGET  := $(BIN_DIR)/example

CC      := gcc
CFLAGS  := -Werror -Wall -Wextra -Wpedantic -I $(HDR_DIR)

SOURCES := $(shell find $(SRC_DIR) -type f -name '*.c')
OBJS    := $(patsubst %.c,$(OUT_DIR)/%.o,$(notdir $(SOURCES)))

vpath %.c $(sort $(dir $(SOURCES)))

all: $(TARGET)

$(TARGET): $(OBJS) | $(BIN_DIR)
        $(CC) $^ -o $@ $(CFLAGS)

$(OUT_DIR)/%.o: %.c | $(OUT_DIR)
        $(CC) $(CFLAGS) -c $^ -o $@

clean:
        rm -f $(TARGET) $(OBJS)

$(OUT_DIR) $(BIN_DIR):
        mkdir -p $@

vpath specifies a list of directories in which make should search a particular class of file names. We use it for the %.c class of file names, and pass it the list of directories in which source files can be found, de-duplicated with sort (sort sorts but also removes duplicates).

Note: you could do the same with the VPATH variable with:

VPATH := $(sort $(dir $(SOURCES)))

but it is less selective (it applies to all classes of file names).

Finally, you could also use foreach-eval-call to programmatically instantiate all compilation rules:

SRC_DIR := src/main
HDR_DIR := src/headers
OUT_DIR := out
BIN_DIR := $(OUT_DIR)/bin
TARGET  := $(BIN_DIR)/example

CC      := gcc
CFLAGS  := -Werror -Wall -Wextra -Wpedantic -I $(HDR_DIR)

SOURCES := $(shell find $(SRC_DIR) -type f -name '*.c')
OBJS    := $(patsubst %.c,$(OUT_DIR)/%.o,$(notdir $(SOURCES)))

all: $(TARGET)

$(TARGET): $(OBJS) | $(BIN_DIR)
        $(CC) $^ -o $@ $(CFLAGS)

# $1: source file
define MY_RULE
$1-object-file := $$(OUT_DIR)/$$(patsubst %.c,%.o,$$(notdir $1))

$$($1-object-file): $1 | $$(OUT_DIR)
        $$(CC) $$(CFLAGS) -c $$^ -o $$@
endef
$(foreach s,$(SOURCES),$(eval $(call MY_RULE,$s)))

clean:
        rm -f $(TARGET) $(OBJS)

$(OUT_DIR) $(BIN_DIR):
        mkdir -p $@

In my opinion, while foreach-eval-call is one of the most powerful GNU make features, this last solution is uselessly complicated in your case, not portable to other makes, and difficult to maintain if you do not know GNU make well: one day or another somebody could think that these $$ are typos, and replace them with $. So, I really think the first solution is preferable, followed by vpath or VPATH if you absolutely want a flat objects directory, collisions cannot happen and portability is not an issue.