Why is this Clojure program so slow? How to make it run fast?

Question

Here it is clearly explained how to optimize a Clojure program dealing with primitive values: use type annotations and unchecked math, and it will run fast:

(set! *unchecked-math* true)

(defn add-up ^long [^long n]
  (loop [n n i 0 sum 0]
    (if (< n i)
      sum
      (recur n (inc i) (+ i sum)))))

So, just out of curiosity, I've tried it in lein repl and, to my surprise, found this code running ~20 times slower that expected (Clojure 1.6.0 on Oracle JDK 1.8.0_11 x64):

user=> (time (add-up 1e8))
"Elapsed time: 2719.188432 msecs"
5000000050000000

Equivalent code in Scala 2.10.4 (same JVM) runs in ~90ms:

def addup(n: Long) = { 
  @annotation.tailrec def sum(s: Long, i: Long): Long = 
    if (i == 0) s else sum(s + i, i - 1)
  sum(0, n)
}

So, what am I missing in the Clojure code sample? Why is it so slow (should theoretically be roughly the same speed)?

Answer 1

Benchmarking with lein repl is generally a bad idea as it specifically sets non-server JVM settings. Using the Clojure JAR directly I see ~40ms on a 3.5ghz i7 iMac running JDK 8 under OS X 10.9.

Answer 2

Further to @dnolen's answer, a few observations:

Though it turns out to make no real difference, we should make the Clojure function the same shape as the Scala one. In

(defn add-up ^long [^long n]
  (loop [n n i 0 sum 0]
    (if (< n i)
      sum
      (recur n (inc i) (+ i sum)))))

• n is not changed by the recur, so need not be bound in the loop.
• The two remaining arguments are in the opposite order to the Scala function.
• It runs up the numbers, whereas the Scala runs down.

Mending these inconsistencies, we get

defn add-up [^long n]
  (loop [sum 0, i n]
    (if (zero? i)
      sum
      (recur (+ sum i) (dec i)))))

(The Scala type system ensures that the argument n is converted to a Long on call. As I understand it (please correct me if I'm wrong), the Clojure ^long type hint promises to treat a Long argument well, but does not promise to convert a Double like 1e8 to a Long. But I got very inconsistent results when I made the corresponding changes.)

On my laptop, the above gives

(time (add-up 100000000))
"Elapsed time: 103.636782 msecs"
5000000050000000

If you remove the type hint

(defn add-up [n]
  ...
  )

... the elapsed time multiplies by about twenty:

(time (add-up 100000000))
"Elapsed time: 2374.399915 msecs"
5000000050000000

• This has more effect than removing unchecked math, which roughly triples the elapsed time.
• Type hinting the return type has no discernible effect.

All this on Clojure 1.5.0 on OpenJDK Java 7.