What is the difference between a variable and a symbol in LISP?

Question

In terms of scope? Actual implementation in memory? The syntax? For eg, if (let a 1) Is 'a' a variable or a symbol?

Answer 1

Jörg's answer points in the right direction. Let me add a bit to it.

I'll talk about Lisps that are similar to Common Lisp.

Symbols as a data structure

A symbol is a real data structure in Lisp. You can create symbols, you can use symbols, you can store symbols, you can pass symbols around and symbols can be part of larger data structures, for example lists of symbols. A symbol has a name, can have a value and can have a function value.

So you can take a symbol and set its value.

(setf (symbol-value 'foo) 42) 

Usually one would write (setq foo 42), or (set 'foo 42) or (setf foo 42).

Symbols in code denoting variables

But!

(defun foo (a)   (setq a 42)) 

or

(let ((a 10))    (setq a 42)) 

In both forms above in the source code there are symbols and a is written like a symbol and using the function READ to read that source returns a symbol a in some list. But the setq operation does NOT set the symbol value of a to 42. Here the LET and the DEFUN introduce a VARIABLE a that we write with a symbol. Thus the SETQ operation then sets the variable value to 42.

Lexical binding

So, if we look at:

(defvar foo nil)  (defun bar (baz)   (setq foo 3)   (setq baz 3)) 

We introduce a global variable FOO.

In bar the first SETQ sets the symbol value of the global variable FOO. The second SETQ sets the local variable BAZ to 3. In both case we use the same SETQ and we write the variable as a symbol, but in the first case the FOO donates a global variable and those store values in the symbol value. In the second case BAZ denotes a local variable and how the value gets stored, we don't know. All we can do is to access the variable to get its value. In Common Lisp there is no way to take a symbol BAZ and get the local variable value. We don't have access to the local variable bindings and their values using symbols. That's a part of how lexical binding of local variables work in Common Lisp.

This leads for example to the observation, that in compiled code with no debugging information recorded, the symbol BAZ is gone. It can be a register in your processor or implemented some other way. The symbol FOO is still there, because we use it as a global variable.

Various uses of symbols

A symbol is a data type, a data structure in Lisp.

A variable is a conceptual thing. Global variables are based on symbols. Local lexical variables not.

In source code we write all kinds of names for functions, classes and variables using symbols.

There is some conceptual overlap:

(defun foo (bar) (setq bar 'baz)) 

In the above SOURCE code, defun, foo, bar, setq and baz are all symbols.

DEFUN is a symbol providing a macro. FOO is a symbol providing a function. SETQ is a symbol providing a special operator. BAZ is a symbol used as data. Thus the quote before BAZ. BAR is a variable. In compiled code its symbol is no longer needed.

Answer 2

Quoting from the Common Lisp HyperSpec:

symbol n. an object of type symbol.

variable n. a binding in the “variable” namespace.

binding n. an association between a name and that which the name denotes. (…)

Explanation time.

What Lisp calls symbols is fairly close to what many languages call variables. In a first approximation, symbols have values; when you evaluate the expression x, the value of the expression is the value of the symbol x; when you write (setq x 3), you assign a new value to x. In Lisp terminology, (setq x 3) binds the value 3 to the symbol x.

A feature of Lisp that most languages don't have is that symbols are ordinary objects (symbols are first-class objects, in programming language terminology). When you write (setq x y), the value of x becomes whatever the value of y was at the time of the assignment. But you can write (setq x 'y), in which case the value of x is the symbol y.

Conceptually speaking, there is an environment which is an association table from symbols to values. Evaluating a symbol means looking it up in the current environment. (Environments are first-class objects too, but this is beyond the scope of this answer.) A binding refers to a particular entry in an environment. However, there's an additional complication.

Most Lisp dialects have multiple namespaces, at least a namespace of variables and a namespace of functions. An environment can in fact contain multiple entries for one symbol, one entry for each namespace. A variable, strictly speaking, is an entry in an environment in the namespace of variables. In everyday Lisp terminology, a symbol is often referred to as a variable when its binding as a variable is what you're interested in.

For example, in (setq a 1) or (let ((a 1)) ...), a is a symbol. But since the constructs act on the variable binding for the symbol a, it's common to refer to a as a variable in this context.

On the other hand, in (defun a (...) ...) or (flet ((a (x) ...)) ...), a is a also symbol, but these constructs act on its function binding, so a would not be considered a variable.

In most cases, when a symbol appears unquoted in an expression, it is evaluated by looking up its variable binding. The main exception is that in a function call (foo arg1 arg2 ...), the function binding for foo is used. The value of a quoted symbol 'x or (quote x) is itself, as with any quoted expression. Of course, there are plenty of special forms where you don't need to quote a symbol, including setq, let, flet, defun, etc.