Macro with &key and &body parameters requires specifying key parameter

I have the following macro:

(defparameter *current-state* 'foo)

(defmacro when-state ((state &key (test #'eq)) &body body)
  `(when (funcall ,test *current-state* ,state)
     ,@body))

Using it in the REPL it works fine with or without specifying key parameter.

However, when using it in a top-level defun or lambda variable declaration, like so:

(defun foo ()
  (when-state ('foo)
    (format t "Foo")))

Then the file won't compile with SBCL with the following error:

; in: DEFUN FOO
;     (SENTO.FSM:WHEN-STATE ('SENTO.FSM::FOO)
;       (FORMAT T "Foo"))
; --> IF FUNCALL 
; ==>
;   1
; 
; caught ERROR:
;   Objects of type COMPILED-FUNCTION can't be dumped into fasl files.
; 
; note: The first argument never returns a value.

Only when explicitly specifying the key parameter it will compile fine.

What is the problem here?

Solution

See the difference:

CL-USER 68 > (macroexpand-1 '(when-state ('foo)
                               (format t "Foo")))
(WHEN (FUNCALL #<Function EQ 80E003BDE9>
               *CURRENT-STATE*
               (QUOTE FOO))
  (FORMAT T "Foo"))
T

In above generated source code, the function EQ is included as an object. This can't be dumped.

Now specify the test in the source:

CL-USER 69 > (macroexpand-1 '(when-state ('foo :test #'eq)
                               (format t "Foo")))
(WHEN (FUNCALL (FUNCTION EQ)
               *CURRENT-STATE*
               (QUOTE FOO))
  (FORMAT T "Foo"))
T

Now quote the test in the WHEN-STATE definition:

CL-USER 70 > (defmacro when-state ((state &key (test '#'eq)) &body body)
               `(when (funcall ,test *current-state* ,state)
                  ,@body))
WHEN-STATE

CL-USER 71 > (macroexpand-1 '(when-state ('foo)
                               (format t "Foo")))
(WHEN (FUNCALL (FUNCTION EQ)
               *CURRENT-STATE*
               (QUOTE FOO))
  (FORMAT T "Foo"))
T

That looks better...

The problem of including function objects in source code

The File Compiler compiles a source function to some other format: C, Byte Code or Machine Code. If the source code includes a function object (for example a compiled function), then this function object would need to be put into the compiled file. The File Compiler would need a way to store function objects in compiled code.

The File Compiler compiles a function to code, but it does not execute the function.

If we look at a macro, then the macro is a part of the compile time environment. Remember: the role of the macro is to generate code, typically at compile time. If the macro generates code, one should not include function objects into the code, but function source code. (function sin) is source code, an evaluated version of that would be the function object: (type-of (eval '(function sin))) -> COMPILED-FUNCTION.

Now if the macro is used in a file, then the file compiler calls the macro function and the returned code is the new source code. This source code should not contain objects, which can not be dumped. Objects which usually can't be dumped are function objects, because they can be a closure or contain executable code.

So:

compiling a function definition does not execute the function.
compiling a macro form, for example in a function definition, executes the the macro function -> don't generate code which includes actual function objects, since they cannot be dumped to the compiled file

Including function objects via reader macros is a similar problem

Another way to include actual functions in code would be to use a reader macro.

(defun foo (a)
  (funcall #.#'eql a 10))

That creates the same problem: the code contains an actual function object, which then creates a problem for the File Compiler, since it can't dump function objects to the compiled file.

Thus we usually write:

(defun foo (a)
  (funcall #'eql a 10))

The reader and the compiler don't evaluate #'eql to a compiled function object.

The reader reads #.#'eql to a compiled function object. -> this should be avoided, just like we avoid to write a macro, which returns compiled function objects.