Possible improvements in clojure "official" concurrency example (using locks,atoms,stm)

Question

i am trying to get the "official" example of clojure concurrency closer to a java version using manual locking. In this gist i put the java and clojure code and the thread dump of a VisualVm profile of all versions. Here it is the clojure code and timing

(ns simple-example (:gen-class))
(set! *warn-on-reflection* true)
;; original from: http://clojure.org/concurrent_programming
(import '(java.util.concurrent Executors Future)
SimpleLocking$Node)

(defn test-concur [iter refs nthreads niters]
  (let [pool (Executors/newFixedThreadPool nthreads)
        tasks (map (fn [t]
                      (fn []
                        (dotimes [n niters]
                          (iter refs t))))
                   (range nthreads))]
    (doseq [^Future future (.invokeAll pool tasks)]
      (.get future))
    (.shutdown pool)))

(defn test-stm [nitems nthreads niters]
  (let [refs (vec (map ref (repeat nitems 0)))
        iter #(dosync (doseq [r %] (alter r + 1 %2)))]
    (test-concur iter refs nthreads niters)
    (map deref refs)))

(defn test-atom [nitems nthreads niters]
  (let [refs (vec (map atom (repeat nitems 0)))
        iter #(doseq [r %] (swap! r + 1 %2))]
    (test-concur iter refs nthreads niters)
    (map deref refs)))

;; SimpleLocking$Node is the class with the synchronized method of java version
(defn test-locking [nitems nthreads niters]
  (let [refs (->> (repeatedly #(SimpleLocking$Node.))
                    (take nitems) vec)
        iter #(doseq [^SimpleLocking$Node n %] (.sum n (+ 1 %2)))]
    (test-concur iter refs nthreads niters)
    (map (fn [^SimpleLocking$Node n] (.read n)) refs)))

(definterface INode
  (read [])
  (add [v]))

(deftype Node [^{:unsynchronized-mutable true} value]
  INode
  (read [_] value)
  (add [this v] (set! value (+ value v))))

(defn test-locking-native [nitems nthreads niters] 
  (let [refs (->> (repeatedly #(Node. 0))
          (take nitems) vec) 
    iter #(doseq [^Node n %]
          (locking n (.add n (+ 1 %2))))]
    (test-concur iter refs nthreads niters)
    (map (fn [^Node n] (.read n)) refs)))

(defn -main [& args]
  (read-line)
  (let [[type nitems nthreads niters] (map read-string args)
    t #(apply + (time (% nitems nthreads niters)))]
    (case type
      'lock (println "Locking:" (t test-locking)) 
      'atom (println "Atom:" (t test-atom))
      'stm (println "STM:" (t test-stm))
      'lock-native (println "Native locking:" (t test-locking-native)))))

Time (in an "old" intel core duo):

Java version
int nitems=100;
int nthreads=10;
final int niters=1000;
Sum node values: 5500000
Time: 31

simple-example=> (-main "lock" "100" "10" "1000")
"Elapsed time: 60.030324 msecs"
Locking: 5500000
nil
simple-example=> (-main "atom" "100" "10" "1000")
"Elapsed time: 202.309477 msecs"
Atom: 5500000
nil
simple-example=> (-main "stm" "100" "10" "1000")
"Elapsed time: 1830.568508 msecs"
STM: 5500000
nil
simple-example=> (-main "lock-native" "100" "10" "1000")
"Elapsed time: 159.730149 msecs"
Native locking: 5500000
nil

NOTE: I dont want get a clojure version as fast as java one, or a stm version as fast as clojure using locks one. I know that is in general difficult and with some problems impossible. I know the use of atoms and stm is more composable,easier to use and less error prone than using manual locks. Those version are only the best possible referents in java and clojure for the problem (well i did my best). My objective is get the atom and stm versions closer to locking ones, or to understand why (maybe in this concrete example) is not possible to speed up those versions.

NOTE: Another comparation, this time with haskell versions using STM and MVars (code in same gist linked):

>SimpleExampleMVar 100000 1000 6
Starting...
2100000000
Computation time: 11.781 sec
Done.

>SimpleExampleSTM 100000 1000 6
Starting...
2100000000
Computation time: 53.797 sec
Done.

>java -cp classes SimpleLocking
Sum node values: 2100000000
Time: 15.703 sec

java -cp classes;%CLOJURE_JAR% simple_example lock 1000 6 100000
"Elapsed time: 27.545 secs"
Locking: 2100000000

java -cp classes;%CLOJURE_JAR% simple_example lock-native 1000 6 100000
"Elapsed time: 80.913 secs"
Native locking: 2100000000

java -cp classes;%CLOJURE_JAR% simple_example atom 1000 6 100000
"Elapsed time: 95.143 secs"
Atom: 2100000000

java -cp classes;%CLOJURE_JAR% simple_example stm 1000 6 100000
"Elapsed time: 990.255 secs"
STM: 2100000000

Answer 1

You are not really comparing like with like here - the Clojure versions are creating and swapping in new immutable boxed numbers whereas the Java version is just bumping a mutable primitive int counter in a synchronised method.

You can do normal Java-style manual locking in Clojure with something like:

(locking obj (set! (. obj fieldName) (+ 1 (.fieldName obj)))))

The locking construct is effectively equivalent to a Java synchronized code block.

If you do this with either a type-hinted Java object or a Clojure deftype with an :unsynchronized-mutable field then I think you should be able to match pure Java synchronized performance.

Haven't tested this but I think it should work with primitives as well (which might be useful if you are incrementing long counters etc.)