Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to set the number of threads to use for par

I know that you can set the number of threads to use for all .par operations like so: collection.parallel.ForkJoinTasks.defaultForkJoinPool.setParallelism(parlevel: Int)

But is it possible to set the number of threads to use for just one .par call?

like image 288
dave Avatar asked Feb 06 '12 01:02

dave


2 Answers

In Scala 2.11 you should use parallel collection task support, like this:

parallelCollection.tasksupport = new ForkJoinTaskSupport(
    new java.util.concurrent.ForkJoinPool(parlevel))

parallelCollection.map( ... )

See task support documentation

like image 83
caiiiycuk Avatar answered Oct 12 '22 15:10

caiiiycuk


You could create a block that sets the parallelism level and then execute specific methods within that block:

def withParallelism[A](n : Int)(block : => A) : A = {
  import collection.parallel.ForkJoinTasks.defaultForkJoinPool._
  val defaultParLevel = getParallelism
  setParallelism(n)
  val ret = block
  setParallelism(defaultParLevel)
  ret
}

And then call it as such:

withParallelism(2) {
  (1 to 100).par.map(_ * 2)
}
like image 41
Submonoid Avatar answered Oct 12 '22 13:10

Submonoid