Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a way to control the number of threads used with pmap?

Tags:

clojure

I am just doing some performance testing with clojure using pmap and I would like to be able to control the number of threads being used with pmap. I know when using something like OpenMP one can set the number of threads using omp_set_num_threads(). I was wondering if there would be anything similar in clojure.

like image 713
Ross Goddard Avatar asked Dec 16 '10 01:12

Ross Goddard


1 Answers

Here's code for pmap:

(defn pmap
  "Like map, except f is applied in parallel. Semi-lazy in that the
  parallel computation stays ahead of the consumption, but doesn't
  realize the entire result unless required. Only useful for
  computationally intensive functions where the time of f dominates
  the coordination overhead."
  ([f coll]
   (let [n (+ 2 (.. Runtime getRuntime availableProcessors))
         rets (map #(future (f %)) coll)
         step (fn step [[x & xs :as vs] fs]
                (lazy-seq
                 (if-let [s (seq fs)]
                   (cons (deref x) (step xs (rest s)))
                   (map deref vs))))]
     (step rets (drop n rets))))

As you can see, pmap takes all available processors and uses them cyclically. So, no, there's no way to set the number of threads... but you can always write your own pmap, which will provide such functionality.

like image 187
ffriend Avatar answered Nov 07 '22 14:11

ffriend