Is 'for' not actually lazy in clojure?

Question

(take 2 (for [x (range 10)
              :let [_ (println x)]
              :when (even? x)] x))
>> (* 0
* 1
* 2
* 3
* 4
* 5
* 6
* 7
* 8
* 9
0 2)

I assumed I was just being remarkably dense. But no, it turns out that Clojure actually evaluates the first 32 elements of any lazy sequence (if available). Ouch.

I had a for with a recursive call in the :let. I was very curious as to why computation seemed to be proceeding in a breadth first rather than depth first fashion. It seems that computation (although, to be fair, not memory) was exploding as I kept going down all the upper branches of the recursive tree. Clojure's 32-chunking was forcing breadth first evaluation, even though the logical intent of the code was depth first.

Anyway, is there any simple way to force 1-chunking rather than 32-chunking of lazy sequences?

Answer 1

Michaes Fogus has written a blog entry on disabling this behavior by providing a custom ISeq implementation.

To steal shamelessly from the modified version by Colin Jones:

(defn seq1 [#^clojure.lang.ISeq s]
  (reify clojure.lang.ISeq
    (first [_] (.first s))
    (more [_] (seq1 (.more s)))
    (next [_] (let [sn (.next s)] (and sn (seq1 sn))))
    (seq [_] (let [ss (.seq s)] (and ss (seq1 ss))))
    (count [_] (.count s))
    (cons [_ o] (.cons s o))
    (empty [_] (.empty s))
    (equiv [_ o] (.equiv s o))))

A simpler approach is given in The Joy of Clojure:

(defn seq1 [s]
  (lazy-seq
    (when-let [[x] (seq s)]
       (cons x (seq1 (rest s))))))

Answer 2

To answer the question in your title, no, for is not lazy. However, it:

Takes a vector of one or more binding-form/collection-expr pairs, each followed by zero or more modifiers, and yields a lazy sequence of evaluations of expr.

(emphasis mine)

So what's going on?

basically Clojure always evaluates strictly. Lazy seqs basically use the same tricks as python with their generators etc. Strict evals in lazy clothes.

In other words, for eagerly returns a lazy sequence. Which won't be evaluated until you ask for it, and will be chunked.