Why does clojure do worse than scala in the alioth benchmarks

Question

See http://shootout.alioth.debian.org/u32q/compare.php?lang=clojure Clojure is much slower than java -server, whereas scala isn't.

What gives?

ref: On Performance and Java Interoperability: Clojure vs. Scala

Answer 1

You can write fast or slow code in any language :-)

Based on a quick inspection of some of the Clojure code, I would say that the main reason for the performance difference is that the Clojure benchmark code hasn't yet been fully optimised to use the fastest language features available.

For example the following features in Clojure are all very cool and useful for development convenience, but incur some runtime performance overhead:

• Lazy sequences and lists
• Dynamic Java interoperability using reflection
• Runtime function composition / first class functions
• Multimethods / dynamic dispatch
• Dynamic compilation with eval or on the REPL
• BigInteger arithmetic

If you want absolute maximum performance (at the cost of some extra complexity), you would want to rewrite code to avoid these and use things like:

• Static type hinting (to avoid reflection)
• Transients
• Macros (for compile time code manipulation)
• Protocols
• Java primitives and arrays
• loop / recur for iteration

With judicious use of the above, I have found that it is generally possible to get very close to Java performance in Clojure 1.2+, e.g. consider the following code to do one million additions:

Unoptimised Clojure using a lazy sequence and biginterger arithmetic. It's nice and functional but it's not particularly fast:

(reduce 
  (fn [acc val] (unchecked-int (unchecked-add (int acc) (int val)))) 
  (range 0 1000000))

=> "Elapsed time: 65.201243 msecs"

Optimized Clojure with primitive arithmetic and loop / recur:

(loop [acc (int 0) i (int 0)] 
  (if (>= i (int 1000000)) 
    acc 
    (recur (unchecked-add acc i) (unchecked-inc i)) ))

=> "Elapsed time: 0.691474 msecs"

Java code, a pretty standard iterative loop:

public static int addMillion() {
    int result=0;
    for (int i=0; i<1000000; i++) {
        result+=i;
    }
    return result;
}

=> "Elapsed time: 0.692081 msecs"

p.s. I have used unchecked-add rather than + in the Clojure code so that it matches the integer overflow behaviour of Java.

Answer 2

Clojure is a dynamic language, it has to do a bunch of extra runtime checking.

Edit, now in a less flippant mood: Without looking at the actual Clojure implementations of the benchmarks, the performance difference can probably also be chalked up to the fact that idiomatic Clojure code eschews mutable state, and, other things being equal, algorithms that use mutable state often outperform those that don't.

Whether a given programmer is able to write code that uses mutable state correctly is a different question, and of course the declarative nature of purely functional code means that a lot of optimizations become possible, in theory at least.

Answer 3

Part of this is due to the test itself, namely that you have to write your code (whatever language it may be) to do the exact same implementation steps as the Java version. Oh, you have a different, possibly faster way to do X in your language? Too bad.

When the basis for the test is for loops and array bashing and a general imperative style requiring multiple floating blocks of code, you're going to get meaningless numbers.

Answer 4

Many of the benchmarks are slower because until very recently it took a lot more work and expert knowledge to get good performance out of Clojure. These benchmarks target mostly 1.2.

Clojure 1.3 has quite a few improvements that makes writing performant code a lot simpler. Even so not as many people in the community are familiar with improving the performance of Clojure code - so it will be a gradual process as people submit better versions.