Why does an elisp local variable keep its value in this case?

Question

Could someone explain to me what's going on in this very simple code snippet?

(defun test-a ()
  (let ((x '(nil)))
    (setcar x (cons 1 (car x)))
    x))

Upon a calling (test-a) for the first time, I get the expected result: ((1)). But to my surprise, calling it once more, I get ((1 1)), ((1 1 1)) and so on. Why is this happening? Am I wrong to expect (test-a) to always return ((1))? Also note that after re-evaluating the definition of test-a, the return result resets.

Also consider that this function works as I expect:

(defun test-b ()
  (let ((x '(nil)))
    (setq x (cons (cons 1 (car x)) 
                  (cdr x)))))

(test-b) always returns ((1)). Why aren't test-a and test-b equivalent?

Answer 1

The Bad

test-a is self-modifying code. This is extremely dangerous. While the variable x disappears at the end of the let form, its initial value persists in the function object, and that is the value you are modifying. Remember that in Lisp a function is a first class object, which can be passed around (just like a number or a list), and, sometimes, modified. This is exactly what you are doing here: the initial value for x is a part of the function object and you are modifying it.

Let us actually see what is happening:

(symbol-function 'test-a)
=> (lambda nil (let ((x (quote (nil)))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1 1))))) (setcar x (cons 1 (car x))) x))

The Good

test-b returns a fresh cons cell and thus is safe. The initial value of x is never modified. The difference between (setcar x ...) and (setq x ...) is that the former modifies the object already stored in the variable x while the latter stores a new object in x. The difference is similar to x.setField(42) vs. x = new MyObject(42) in C++.

The Bottom Line

In general, it is best to treat quoted data like '(1) as constants - do not modify them:

quote returns the argument, without evaluating it. (quote x) yields x. Warning: quote does not construct its return value, but just returns the value that was pre-constructed by the Lisp reader (see info node Printed Representation). This means that (a . b) is not identical to (cons 'a 'b): the former does not cons. Quoting should be reserved for constants that will never be modified by side-effects, unless you like self-modifying code. See the common pitfall in info node Rearrangement for an example of unexpected results when a quoted object is modified.

If you need to modify a list, create it with list or cons or copy-list instead of quote.

See more examples.

PS. This has been duplicated on Emacs.

PPS. See also Why does this function return a different value every time? for an identical Common Lisp issue.

Answer 2

I found the culprit is indeed 'quote. Here's its doc-string:

Return the argument, without evaluating it.

...

Warning: `quote' does not construct its return value, but just returns the value that was pre-constructed by the Lisp reader

...

Quoting should be reserved for constants that will never be modified by side-effects, unless you like self-modifying code.

I also rewrote for convenience

(setq test-a 
      (lambda () ((lambda (x) (setcar x (cons 1 (car x))) x) (quote (nil)))))

and then used

(funcall test-a)

to see how 'test-a was changing.

Answer 3

It looks like the '(nil) in your (let) is only evaluated once. When you (setcar), each call is modifying the same list in-place. You can make (test-a) work if you replace the '(nil) with (list (list)), although I presume there's a more elegant way to do it.

(test-b) constructs a totally new list from cons cells each time, which is why it works differently.