Clojure dothreads! function

Question

I am reading Fogus' book on Joy of Clojure and in the parallel programming chapter I saw a function definition which surely want to illustrate something important but I can't find out what. Moreover, I can't see what is this function for - when I execute, it doesn't do anything:

(import '(java.util.concurrent Executors))
  (def *pool* (Executors/newFixedThreadPool
    (+ 2 (.availableProcessors (Runtime/getRuntime)))))

(defn dothreads! [f & {thread-count :threads
                       exec-count :times
                       :or {thread-count 1 exec-count 1}}]
  (dotimes [t thread-count]
    (.submit *pool* #(dotimes [_ exec-count] (f)))))

I tried to run in this way:

(defn wait [] (Thread/sleep 1000))
(dothreads! wait :thread-count 10 :exec-count 10)
(dothreads! wait)
(dothreads! #(println "running"))

...but it returns nil. Why?

Answer 1

So, here's the same code, tweaked slightly so that the function passed to dothreads! gets passed the count of the inner dotimes.

(import 'java.util.concurrent.Executors)

(def ^:dynamic *pool* (Executors/newFixedThreadPool (+ 2 (.availableProcessors (Runtime/getRuntime)))))

(defn dothreads! [f & {thread-count :threads
                       exec-count :times
                       :or {thread-count 1 exec-count 1}}]
  (dotimes [t thread-count]
    (.submit *pool* #(dotimes [c exec-count] (f c)))))

(defn hello [name]
  (println "Hello " name))

Try running it like this:

(dothreads! hello :threads 2 :times 4)

For me, it prints something to the effect of:

Hello  0
Hello  1
Hello  2
Hello  3
nil
user=> Hello  0
Hello  1
Hello  2
Hello  3

So, note one mistake you made when calling the function: you passed in :thread-count and :exec-count as the keys whereas those are actually the bindings in the destructuring that's happening inside dothreads!. The keywords are the words starting with a colon, :threads and :times.

As to what this code actually does:

1. It creates a new fixed size thread pool that will use at most the number of cores in your machine + 2. This pool is called *pool* and is created using the Java Executor Framework. See [1] for more details.

2. The dothreads! function gets a function that will be called exec-count times on each of the thread-count threads. So, in the example above, you can clearly see it being called 4 times per thread (:threads being 2 and :times being 4).

3. The reason why this function returns nil is that the function dothreads! doesn't return anything. The submit method of the thread pool returns void in Java and this means it returns nil in Clojure. If you were to add some other expression at the end of the function making it:

   (defn dothreads! [f & {thread-count :threads
                          exec-count :times
                          :or {thread-count 1 exec-count 1}}]
     (dotimes [t thread-count]
       (.submit *pool* #(dotimes [c exec-count] (f c))))
     (* thread-count exec-count))
   

It will return 8 for the example above (2 * 4). Only the last expression in the function is returned, so if in a function you were to write (fn [x y] (+ x y) (* x y)) this will always return the product. The sum will be evaluated, but it will be for nothing. So, don't do this! If you want add more that one expression to a function, make sure that all but the last one have side effects, otherwise they'll be useless.

1. You might also notice that the order in which stuff is printed is asynchronous. So, on my machine, it says hello 4 times, then returns the result of the function and then says hello 4 other times. The order in which the functions are executed is undetermined between threads, however the hellos are sequential in each thread (there can never be a Hello 3 before a Hello 2). The reason for the sequentiality is that the function actually submitted to the thread pools is #(dotimes [c exec-count] (f c)) and

[1] http://download.oracle.com/javase/tutorial/essential/concurrency/executors.html

Answer 2

It's used afterwards in the book to run test functions multiple times in multiple threads. It doesn't illustrate anything by itself, but it's used to demonstrate locking, promises, and other parallel and concurrent stuff.