How to Define Makefile Rule for Bulk and Individual Builds

I have a makefile that uses dmake v4.1 for an existing tool. There is a rule for building one file type from another using a % rule like so:

%.html : %.docx
  <recipe lines>

This rule takes Word docs and runs a script to convert them to HTML. However, due to a number of IS infrastructure rules out of my control, it's much slower to run this rule for a number of files separately. The script that handles the conversion is capable of handling multiple files and using a single launch of MS Word to process all documents. I'd like to try to find a way to have a makefile rule that supports this while retaining the rule for converting the file individually.

I have considered modifying the build script and conversion script to launch MS Word beforehand, then using the existing instance to do the file conversion, but I'd like to try to avoid this if possible. At the moment, here's an example of what the rules I have look like:

FILE_LIST = a.html b.html c.html

all .PHONY : $(FILE_LIST)
  
%.html : %.docx
  $(CONVERT_DOC) $<

I've tried having a separate rule for bulk conversion like so (where DOCX_FILE_LIST is just FILE_LIST with the extensions changed):

group_convert .PHONY : $(DOCX_FILE_LIST)
  $(CONVERT_DOC) $^

However it seems that the $^ macro that's supposed to evaluate to the list of out of date pre-requisites always evaluates to the full file list (likely because group_convert is marked with .PHONY). Is it possible to achieve something like I'm looking to do?

Solution

I've tried having a separate rule for bulk conversion like so (where DOCX_FILE_LIST is just FILE_LIST with the extensions changed):
group_convert .PHONY : $(DOCX_FILE_LIST)
  $(CONVERT_DOC) $^
However it seems that the $^ macro that's supposed to evaluate to the list > of out of date pre-requisites always evaluates to the full file list (likely > because group_convert is marked with .PHONY).

dmake's target attributes are idiosyncratic to that make implementation, and somewhat confusing to folks used to GNU make's conflicting use of some of the same symbols. Nevertheless, the significance of a phony target in either make implementation is that it is always out of date at the beginning of a run, regardless of whether a file whose name matches the target exists or what its timestamp is.

That's probably not much relevant to your issue, however. Inasmuch as you mark group_convert phony, I presume that no such file actually exists, and the build system doesn't create one. THAT is why all its prerequistes are always taken to be newer than it. Remember always that make itself has no absolute sense of whether files have changed since a previous run (even dmake's unusual .KEEP_STATE facility does not provide this). make's analysis of whether a target is out of date with respect to its prerequisites is always a relative one.

Is it possible to achieve something like I'm looking to do?

Yes. You can have dmake create a tag file whose purpose is simply to memorialize the timestamp of the last doc conversion. You could do that with a rule similar to what you already have:

group_convert.timestamp: $(DOCX_FILE_LIST)
    $(CONVERT_DOC) $^
    touch $@

Note well that target group_convert.timestamp must not be marked phony. That file is actually created by the build, and its timestamp is used to judge which documents need to be converted.

There are some caveats with that, however:

The timestamp used to judge file staleness is associated with an ancillary file, not with the actual output files. There is therefore the possibility that one of the .docx files is modified during a run of that recipe, in which case it would not have been picked up as a file to convert, but its timestamp will be too old to pick it up for conversion on subsequent runs.
You could avert that by updating the timestamp first, but that would be worse, because then, if the file conversion process fails or is interrupted in the middle (a more likely event) then none of the files that as a result were missed on that conversion would be converted on subsequent ones, until they are modified again.

Another, more involved approach would be to use per-file rules to evaluate which HTML files were out of date with respect to their DOCX sources, but to collect the results in a file instead of immediately updating (this is a little nasty relative to the normal mode of operation of make, but it could be made to work). Then do have a .PHONY group_convert target. Let it use the contents of the previously created file to control which documents need to be converted. That might look something like this:

CONVERT_FILE = newer_docx_files
FILE_LIST = a.html b.html c.html

# This target is phony, so it always starts out of date, even though
# the recipe does create a file with the target's name:
$(CONVERT_FILE) .PHONY :
    echo -n > $(CONVERT_FILE)
    $(MAKECMD) $(MAKEFILE) $(MAKEFLAGS) $(FILE_LIST)

# This target is phony, so it always starts out of date:
group_convert .PHONY : $(CONVERT_FILE)
    to_convert=$$(<$<); \
    test -n "$${to_convert}" && $(CONVERT_DOC) $${to_convert}

# This rule is a lie.  It does not actually create .html files from
# corresponding .docx files, but it *does* capture the result of the
# staleness analysis to control whether the .html file is built later.
%.html : %.docx
    echo '$<' >> $(CONVERT_FILE)