Real world Clojure performance tuning tips?

Question

I am seeing some serious slowdowns in "some" parts of a Clojure application. Does anyone have any real world tips as to how they have done performance tuning on a clojure application?

Answer 1

My personal tips:

• Check your algorithm first - are you incurring O(n^2) cost when it really should be O(n.log n)? If you've picked a bad algorithm, the rest of tuning is a waste of time.
  • Be aware of common "gotchas" like the O(n) cost of traversing a list / sequence.
  • Take advantage of nice features in Clojure, like the O(1) cost of copying a large persistent data structure or the O(log32 n) cost of map/set/vector accesses.
• Choose among Clojure's core constructs wisely:
  • An atom is great when you need some mutable data, e.g. updating some data in a loop
  • If you are going to traverse some data in sequence, use a list rather than a vector or map since this will avoid creating temporary objects while traversing the sequence.
  • Use deftype/defrecord/defprotocol where appropriate. These are heavily optimised, and in particular should be preferred to defstruct/multimethods as of Clojure 1.2 onwards.
• Take advantage of Clojure's concurrency capabilities:
  • pmap and future are both relatively easy ways to take advantage of multiple cores when you are doing a number of independent computations at the same time.
  • Remember that because of Clojure's immutable persistent data structures, making and working on multiple copies of data is very inexpensive. You also don't have to worry about locking when taking snapshots.....
• If you are interfacing with Java code, use "(set! *warn-on-reflection* true)" and eliminate every reflection warning. Reflection is one of the most expensive operations, and will really slow your application down if done repeatedly.
• If you still need more performance, identify the most performance critical parts of your code (e.g. the 5% of lines where the application spends 90%+ of CPU time), analyse this section in detail and judiciously apply the following rules:
  • Avoid laziness. Laziness is a great feature but comes with some additional overhead. Be aware that many of Clojure's common sequence / list functions are lazy (e.g. for, map, partition). loop/recur, dotimes and reduce are your non-lazy friends.
  • Use primitive hints and unchecked arithmetic to make arithmetic / numerical code faster. Primitives are much faster than Clojure's default BigInteger arithmetic.
  • Minimise memory allocations - try to avoid creating too much unnecessary intermediate data (vectors, lists, maps, non-primitive numbers etc.). All allocations impose a small amount of extra overhead, and also result in more/longer GC pauses over time (this is likely to be a bigger concern if you are writing a game / soft realtime app.).
  • (Ab)use Java arrays - not really idiomatic in Clojure, but aget / aset / areduce and friends are very fast (they benefit from a lot of JVM optimisations!!). (Ab)use primitive arrays for extra bonus points.
  • Use macros - to generate ugly-but-fast code at compile time where possible

Doing all the above should get pretty good performance out of Clojure code - with careful tuning I've usually been able to get reasonably near to pure Java performance, which is pretty impressive for a dynamic language!

Answer 2

You can use JVisualVM to do profiling of Clojure code (see JVisualVM and Clojure for an example). This should at least point you in the right direction of the slow code.

Answer 3

This question covers profiling with Clojure: Profiling tool for Clojure?

I'm sure you'll find some good tips there.

Then straight from the horse's mouth: http://clojure.org/getting_started#Getting%20Started-Profiling