Why does Clojure xform with side effects not behave as expected?

Question

I've found that adding side effects to a Clojure transducer can result in the following unexpected behavior:

(def counter (atom 0))

(def increment-counter-xform
  (map (fn [sample]
         (swap! counter inc)
         sample)))

(eduction increment-counter-xform (range 100))
@counter ;; => 100, as expected

(reset! counter 0)
(eduction increment-counter-xform (range 1000))
@counter ;; => 129, but I expected 1000

Why does the above code block increment the counter the expected number of times for small numbers of samples, but not for large numbers of samples? It appears that even if the argument to range is very large, the counter will never be incremented more than 129 times.

Answer 1

The problem was that eduction returns a lazy sequence. I think that Clojure has some heuristics for how many terms of a lazy sequence it will realize if not forced to, and in this case it was realizing 129 terms. Everything works as expected if the second-to-last line in the example code is changed to

(vec (eduction increment-counter-xform (range 1000)))

It works because vec forces the entire sequence to be realized.