Clojure "repeatedly" makes "future" run sequentially

Question

While this snippet

(dorun 
  (map deref 
    (map #(future 
            (println % (Thread/currentThread))) 
         (range 10))))

prints 10 intermixed lines showing different threads:

0 #object[java.lang.Thread 0x5f1b4a83 Thread[clojure-agent-send-off-pool-26,5,main]]                                                                                                                           
2 #object[java.lang.Thread 1 0x79dfba1f #object[Thread[clojure-agent-send-off-pool-28,5,main]java.lang.Thread]                                                                                                 
3 4 #object[java.lang.Thread #object[java.lang.Thread 0x7ef7224f Thread[clojure-agent-send-off-pool-27,5,main]0x5f1b4a83 ]Thread[clojure-agent-send-off-pool-26,5,main]]                                       
5                                                                                                                                                                                                              
67  #object[java.lang.Thread #object[0x79dfba1f java.lang.Thread Thread[clojure-agent-send-off-pool-28,5,main]]0x77526645                                                                                      
 8 #object[java.lang.Thread #object[java.lang.ThreadThread[clojure-agent-send-off-pool-29,5,main] ]9 #object[java.lang.Thread 0xc143aa5 0x7ef7224f                                                             Thread[clojure-agent-send-off-pool-31,5,main]]Thread[clojure-agent-send-off-pool-27,5,main]]                                                                                                                       

0x1ce8675f 0x379ae862 Thread[clojure-agent-send-off-pool-30,5,main]Thread[clojure-agent-send-off-pool-32,5,main]]]

as I would expect, the following snippet:

(dorun
  (map deref 
    (map #(future 
            (println % (Thread/currentThread))) 
         (repeatedly 10 #(identity 42)))))

produces 10 neatly aligned strings with the same thread:

42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                                                                                                                              
42 #object[java.lang.Thread 0x1e1b7ffb Thread[clojure-agent-send-off-pool-39,5,main]]                          

which clearly indicates that the futures are not run in parallel, but each in the same thread.

This happens only with repeatedly, even if I realize the sequence with doall first, but vectors, ranges or other sequences all result in parallel execution.

Why is future dispatching to the same thread when repeatedly is used?

Thanks!

Answer 1

This works:

(dorun (map deref (doall (map #(future (println % (Thread/currentThread))) (repeatedly 10 #(identity 42))))))

The problem is that range produces a chunked sequence while repeatedly produces an unchunked sequence. Map is lazy, so in the repeatedly case you're creating a future, then derefing it, then creating the next future, then dereffing it. In the range case the sequence is chunked so you're creating all futures and then derefing all of them.

Here's another fun way to observe the difference between the behaviour of chunked and unchunked sequences.

=> (first (map prn (range 10)))
0
1
2
3
4
5
6
7
8
9
nil
=> (first (map prn (repeatedly 10 #(identity 13))))
13
nil

The size of the chunks is usually 32 (but I think that's not guaranteed anywhere), as can be seen if you run (first (map prn (range 1000))).

Chunking is one of those hidden features of Clojure that you usually learn when it first bites you :)