Make macros and functions integrate more seamlessly

Question

OK, I understand pretty well how to use both function and macros.

What I'm curious about is why the compiler can't be a bit more clever when integrating the two, e.g. consider the Clojure code:

(defmacro wonky-add [a b] `(+ ~a (* 2 ~b)))

(defn wonky-increment [a] (apply wonky-add a 1))
=> Error: can't take value of a macro

Yes, I know I can make this work by taking out the "apply" - but why is it that the compiler can't figure out how to do this itself?

Would it be possible in Clojure / other LISPs to create a version of apply or other higher order functions that work equally well with both functions and macros as parameters?

Answer 1

This answer is not so much about Clojure, but Lisp and macros in general.

Remember:

APPLY is there to be able to call functions with argument lists that are created at runtime.

Macros are there to generate new source code from some source code - and the generated source code will run.

Now:

If you allow apply to feed different source code into a macro at runtime, one needs to be able to generate the result code at runtime and also to execute the generated code.

Thus in a compiled Lisp system, every call of APPLY with a macro potentially creates new code that needs to be compiled at runtime to be able to execute it. This means also that you may get new compiler errors at runtime, when your code is executing and the code has some problem. So to APPLY macros at runtime, you need a compiler and all necessary information (for example other macros) to be able to expand and compile the code.

This also means that in the general case you can't compile APPLY with a macro before runtime, since it is not known what the runtime arguments to apply will be.

In an interpreter-based Lisp, there may be more information and macros can be applied all the time.

In early Lisp there were normal functions and so-called FEXPRs. FEXPRs allowed flexible calling and runtime code manipulation. Later in the history of Lisp they have been replaced with macros, since macros allow efficient compilation and, in compiler-based systems, allow syntax errors to be handled before runtime.