How to override a included Makefile variable when it is used for dependency list?

I have Makefiles for several custom domain specific languages. One of them require weird paths for the input/output files passed to the compiler, So I had to use some gnumake tricks. Now they are not playing nice with variable overloading, even using the always-re-parse-variable (=).

# Makefile.main
SOURCE_FILES=forgot_to_set_source_files.ksp

# Paths
SRCDIR=src
OBJDIR=Resources/scripts

.SUFFIXES:            # Delete the default suffixes
.SUFFIXES: .txt .ksp  # Define our suffix list
VPATH =
VPATH = $(SRCDIR)
OBJECT_FILES = $(addprefix $(OBJDIR)/, $(SOURCE_FILES:.ksp=.txt))

# src/abc.ksp -> Resources/script/abc.txt
$(OBJECT_FILES): $(OBJDIR)/%.txt: %.ksp
    $(info Compiling $<)
    @mkdir -p $(@D)
    "$(CC)" $(CC_FLAGS) $< $@


build: $(VMPID) $(OBJECT_FILES)
    $(info $(OBJECT_FILES))
    $(info $(SOURCE_FILES))

Now each project have:

# projA/Makefile
include ../Makefile.main

SOURCE_FILES=\
    one.ksp \
    two.ksp

Now when i try to make:

 $ make
make: *** No rule to make target 'forgot_to_set_source_files.ksp', needed by 'Resources/scripts/forgot_to_set_source_files.txt'.  Stop.

tried to remove the default value

$ git diff ../Makefile.main
- SOURCE_FILES=forgot_to_set_source_files.ksp
+ #SOURCE_FILES

$ make
Resources/scripts/one.txt Resources/scripts/two.txt
one.ksp two.ksp
make: Nothing to be done for 'default'.

From the output of the two $(info ...) we confirm make "sees" an empty variable for the rules targets, but see the correct value inside the rule.

I then tried to reset all values that are used to construct the rule target into the project Makefile, and added some exports for good measure.

# projA/Makefile
include ../Makefile.main

export SOURCE_FILES=\
    one.ksp \
    two.ksp

export OBJECT_FILES=$(addprefix $(OBJDIR)/, $(SOURCE_FILES:.ksp=.txt))

export

No change. Still exact same failures.

I tried to re-declare the sulfix rule ($(OBJECT_FILES): $(OBJDIR)/%.txt: %.ksp) in the project makefile. Still fails!

I tried to use ?= on the main makefile's assignment of SOURCE_FILES, and I still get No rule to make target 'forgot_to_set_source_files.ksp'...

What is preventing that rule from getting the updated value of SOURCE_FILES? Is there anything special about $(addprefix ...)?

Solution

I'm not sure I understood everything, but you should definitely read How make Reads a Makefile and be sure you understand where things are expanded.

All variable references that appear as part of a recipe target or prerequisite list are expanded immediately when that rule is parsed. So in this rule:

# Makefile.main
SOURCE_FILES=forgot_to_set_source_files.ksp

$(OBJECT_FILES): $(OBJDIR)/%.txt: %.ksp
    ...

the references to OBJECT_FILES (which includes a reference to SOURCE_FILES) and OBJ_DIR are expanded right here. That happens before the next line of the makefile is parsed, and long before the value of SOURCE_FILES is reset later by the included makefile.

If you want this recipe to use the updated versions of these variables you must delay the definition of the rule until after the final values are set; that is, you need to put them at the end of the projA/Makefile, not at the beginning.

Often people will either put their include of "main" at the end or else, if that won't work, use two included files one for common variable assignments at the beginning and a second for rule definitions at the end.