Non-recursive symbol-macrolet

Question

I'd like to expand all symbols x in a certain fragment into (value x). E.g.

(lambda ()
  (* x x))

should become

(lambda ()
  (* (value x) (value x)))

A simple use of symbol-macrolet wouldn't work, because

(symbol-macrolet ((x (value x)))
  (lambda ()
    (* x x)))

explodes into infinite recursion during macro expansion as the result of a symbol-macrolet expansion is processed again for macro expansions, including the same symbol-macrolet.

Even trying to expand x to (value y) and then y to x doesn't work. For example:

(symbol-macrolet ((y x))
  (symbol-macrolet ((x (value y)))
    (lambda () (* x x))))

still crashes at macroexpansion time on SBCL.

Is there a way to expand the symbol once only without doing a full code walking?

Answer 1

This was discussed in a comp.lang.lisp thread from 2013. A user jathd pointed out:

The behaviour you observe comes from the way macroexpansion works in general: when processing a form for evaluation (or compilation, or macroexpansion, or...), if it is a macro invocation, replace it with the corresponding expansion and start the processing from the start with the new form.

So you would have to actively do something special for symbol macros, as opposed to regular macros, for them not to be "recursive".

Pascal Constanza provided this advice:

A good solution is to make the symbol macro expand into a regular macro.

(macrolet ((regular-macro (...) ...)) 
   (symbol-macrolet ((sym (regular-macro))) 
     ...))

although informatimago points out that this still exhibits the same behavior that the original did:

That would still fail if the regular macro expansion included in a macroexpandable place the symbol naming the symbol macrolet.

The answer to “Is there a way to expand the symbol once only without doing a full code walking?” seems to be "no," unfortunately. However, it's not too hard to work around this; the "solution" in the linked thread ends up using gensyms to avoid the problem. E.g.:

(let ((x 32))                            ; just to provide a value for x
  (let ((#1=#:genx x))                   ; new variable with x's value
    (symbol-macrolet ((x (values #1#)))  ; expansion contains the new variable
      (* x x))))                         ; === (* (values #1#) (values #1#))
;=> 1024

Writing #1# or similar things in the macroexpansion isn't fun though. It's not too bad if you're automatically generating the expansions, but if you're doing this manually, it might be useful to exploit the fact that let can shadow symbol-macrolet. This means that you can wrap your expansion in a let that restores the binding that you want:

(let ((x 32))
  (let ((#1=#:genx x))
    (symbol-macrolet ((x (let ((x #1#))    ; boilerplate
                           (values x))))   ; you get to refer to `x` here
      (* x x))))
;=> 1024

If you find yourself doing this a lot, you can wrap it up in an "unshadowing" version of symbol-macrolet:

(defmacro unshadowing-symbol-macrolet (((var expansion)) &body body)
  "This is like symbol-macrolet, except that var, which should have a binding
   in the enclosing environment, has that same binding within the expansion of
   the symbol macro.  This implementation only handles one var and expansion;
   extending to n-ary case is left as an exercise for the reader."
  (let ((hidden-var (gensym (symbol-name var))))
    `(let ((,hidden-var ,var))
       (symbol-macrolet ((,var (let ((,var ,hidden-var))
                                 ,expansion)))
         ,@body))))

(let ((x 32))
  (unshadowing-symbol-macrolet ((x (values x)))
    (* x x)))
;=> 1024

That will only work for a variable that already has a lexical binding, of course. Common Lisp doesn't provide much in the way of accessing environment objects, aside from passing them around in macro expansions. If your implementation provides environment access, you could have the unshadowing-symbol-macrolet check whether each var is bound in the environment, and provide the local shadowing if it is, and do no shadowing if it isn't.

Notes

It's interesting to see what the original author in that thread, Antsan, had to say about their expectations of how the macroexpansion process worked:

I thought macro expansion worked by repeatedly doing macro expansion on the source until a fixpoint is reached. This way SYMBOL-MACROLET would be automatically non-recursive if it was removed by macro expansion.

Something like:

(symbol-macrolet (a (foo a)) 
  a) 
macroexpand-1> (foo a) 
macroexpand-1> (foo a) ; fixpoint

No special case would be needed there, although I guess that this algorithm for macro expansion would be slower.

It's interesting, because that's exactly how Common Lisp's' compiler macros work. The documentation from define-compiler-macro says:

• Unlike an ordinary macro, a compiler macro can decline to provide an expansion merely by returning a form that is the same as the original (which can be obtained by using &whole).

That wouldn't really help here, since symbol-macros don't get a choice about what to return; that is, there are no arguments passed to a symbol-macro, so there's nothing that can be checked or used to influence what the macroexpansion is. The only way to return the same form would be something like (symbol-macrolet ((x x)) …), which rather defeats the purpose.