setq vs Undeclared free variable (common lisp)

Question

I'm a total Lisp n00b, so please be gentle.

I'm having trouble wrapping my head around CL's idea of an [un-]declared free variable. I would think that:

(defun test ()
    (setq foo 17)
)

would define a function that declares a variable foo and set it to 17. However, instead I get

;Compiler warnings :
;  In TEST: Undeclared free variable FOO

My actual example case is a bit bigger; my code (snippet) looks like this:

(defun p8 ()
    ;;; [some other stuff, snip]

    (loop for x from 0 to (- (length str) str-len) do
        (setq last (+ x str-len))           ; get the last char of substring
        (setq subs (subseq str x last))     ; get the substring
        (setq prod (prod-string subs))      ; get the product of that substring
        (if (> prod max)                    ; if it's bigger than current max, save it
            (setq max prod)
            (setq max-str subs)
        )
    )

;;; [More stuff, snip]
)

and that gives me:

;Compiler warnings for "/path/to/Lisp/projectEuler/p6-10.lisp":
;   In P8: Undeclared free variable LAST (2 references)
;Compiler warnings for "/Volumes/TwoBig/AllYourBits-Olie/WasOnDownBelowTheOcean/zIncoming/Lisp/projectEuler/p6-10.lisp" :
;   In P8: Undeclared free variable PROD (3 references)
;Compiler warnings for "/Volumes/TwoBig/AllYourBits-Olie/WasOnDownBelowTheOcean/zIncoming/Lisp/projectEuler/p6-10.lisp" :
;   In P8: Undeclared free variable SUBS (3 references)
;Compiler warnings for "/Volumes/TwoBig/AllYourBits-Olie/WasOnDownBelowTheOcean/zIncoming/Lisp/projectEuler/p6-10.lisp" :
;   In P8: Undeclared free variable =

Yes, yes, I realize that I'm using far too many intermediate variables, but I'm trying to understand what's going on before I get too fancy with compressing everything down to the minimal typed characters, as seems popular in the CL world.

So, anyway... can someone explain the following:

• Under what conditions does Lisp "declare" a variable?
• Is the scope of said variable other than the enclosing (...) around the setq statement?! (That is, I would expect the var to be valid & scoped for everything from (... (setq ...) ...) the parens 1 level outside the setq, no?
• Am I mis-interpreting the Undeclared free variable message?
• Any other hints you care to give that would help me better grok what's going on, here.

NOTE: I'm fairly expert with C, Java, Javascript, Obj-C, and related procedural languages. I get that functional programming is different. For now, I'm just wrestling with the syntax.

Thanks!

P.S. If it matters, defun p8 is in a text file (TextMate) and I'm running it on Clozure CL. Hopefully, none of that matters, though!

Answer 1

In lisp variables may be declared usingdefparameterordefvar.

(defparameter var1 5)
(defvar var2 42)

This results in global (dynamic) variables.

The difference between defvarand defparameteris that defvardoes not reinitialize an already existing variable.

Local (lexical) variables are introduced e.g using let or let* (which initializes the variables sequentially).

Undeclared free variable means that you have used (here setq-ed) a variable that it is not bound in the context it is used. It may then be declared for you, but then probably as a global (dynamic) variable. The consequence of this is that if you use undeclared variables with the same name in several functions, you will be referencing the same variable in all of the functions.

Your code can be written like this:

(loop for x from 0 to (- (length str) str-len) do
    (let* ((last (+ x str-len))         ; get the last char of substring
           (subs (subseq str x last))   ; get the substring
           (prod (prod-string subs)))   ; get the product of that substring
      (if (> prod max)                    ; if it's bigger than current max, save it
          (setq max prod)
          (setq max-str subs))))

Using the variable-binding properties of loop, it may also be written as

(loop for x from 0 to (- (length str) str-len)
      for last = (+ x str-len)
      for subs = (subseq str x last)
      for prod = (prod-string subs)
      when (> prod max) do
          (setq max prod)
          (setq max-str subs))

Answer 2

In Lisp variable declaration may be performed in many ways. The most notable are:

• declaring global (they are properly called special) variables with defparameter and defvar
• declaring local variables with let, let*, multiple-value-bind, destructuring-bind, and other binding forms
• as function arguments

You can read about their scope in many places as well, for instance in CLtL2.

setq/setf are not variable declaration operators, but variable modification operators, as implied by their names.

PS. In interactive mode some implementation would use the DWIM approach and declare the variable as special behind the scenes, if you try to set an undeclared variable, but this is purely for convenience.

Answer 3

The Common Lisp HyperSpec (basically its the Common Lisp standard in HTML form) says:

http://www.lispworks.com/documentation/HyperSpec/Body/s_setq.htm

Assigns values to variables.

So SETQ only assigns values to variables. It does not declare them.

Variable definitions are done globally with DEFVAR, DEFPARAMETER, ...

(defparameter *this-is-a-global-dynamic-variable* 'yep)

Variable definitions are done locally with DEFUN, LET, LET*, LOOP, and many others.

(defun foo (v1 v2)
  ...)

(let ((v1 10)
      (v2 20))
  ...)

(loop for v1 in '(10 30 10 20)
      do ...)

This is basic Lisp and it would be useful to read an introduction. I would recommend:

http://www.cs.cmu.edu/~dst/LispBook/

Above book is free for download.

Additionally the above mentioned Common Lisp Hyperspec provides you with the definitions for Common Lisp and describes the various facilities (DEFUN, LOOP, DEFPARAMETER, ...) in detail.