Why is there such a difference in speed between Clojure's loop and iterate methods

Question

I have a question about why there is such a difference in speed between the loop method and the iterate method in clojure. I followed the tutorial in http://www.learningclojure.com/2010/02/clojure-dojo-2-what-about-numbers-that.html and defined two square-root methods using the Heron method:

(defn avg [& nums] (/ (apply + nums) (count nums)))
(defn abs [x] (if (< x 0) (- x) x))
(defn close [a b] (-> a (- b) abs (< 1e-10) ) )

(defn sqrt [num]
  (loop [guess 1]
    (if (close num (* guess guess))
        guess
     (recur (avg guess (/ num guess)))
)))

(time (dotimes [n 10000] (sqrt 10))) ;;"Elapsed time: 1169.502 msecs"


;; Calculation using the iterate method
(defn sqrt2 [number]
    (first (filter #(close number (* % %)) 
        (iterate #(avg % (/ number %)) 1.0))))

(time (dotimes [n 10000] (sqrt2 10))) ;;"Elapsed time: 184.119 msecs"

There is about a x10 increase in speed between the two methods and I'm wondering what is happening below the surface to cause the two to be so pronouced?

Answer 1

Your results are surprising: normally loop/recur is the fastest construct in Clojure for looping.

I suspect that the JVM JIT has worked out a clever optimisation for the iterate method, but not for the loop/recur version. It's surprising how often this happens when you use clean functional code in Clojure: it seems to be very amenable to optimisation.

Note that you can get a substantial speedup in both versions by explicitly using doubles:

(set! *unchecked-math* true)

(defn sqrt [num]
  (loop [guess (double 1)]
    (if (close num (* guess guess))
      guess
      (recur (double (avg guess (/ num guess)))))))

(time (dotimes [n 10000] (sqrt 10)))
=> "Elapsed time: 25.347291 msecs"


(defn sqrt2 [number]
  (let [number (double number)]
    (first (filter #(close number (* % %)) 
      (iterate #(avg % (/ number %)) 1.0)))))

(time (dotimes [n 10000] (sqrt 10)))
=> "Elapsed time: 32.939526 msecs"

As expected, the loop/recur version now has a slight edge. Results are for Clojure 1.3