Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can views be used with parallel collections?

Tags:

The idiom for finding a result within a mapping of a collection goes something like this:

list.view.map(f).find(p)

where list is a List[A], f is an A => B, and p is a B => Boolean.

Is it possible to use view with parallel collections? I ask because I'm getting some very odd results:

Welcome to Scala version 2.9.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0).
Type in expressions to have them evaluated.
Type :help for more information.

scala> val f : Int => Int = i => {println(i); i + 10}
f: Int => Int = <function1>

scala> val list = (1 to 10).toList
list: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

scala> list.par.view.map(f).find(_ > 5)
1
res0: Option[Int] = Some(11)

scala> list.par.view.map(f).find(_ > 5)
res1: Option[Int] = None
like image 333
Luigi Plinge Avatar asked Dec 12 '11 06:12

Luigi Plinge


People also ask

Why do we use parallel collection classes in Scala?

Parallel collections were included in the Scala standard library in an effort to facilitate parallel programming by sparing users from low-level parallelization details, meanwhile providing them with a familiar and simple high-level abstraction.

Is Scala parallel programming?

As already said, Scala implements collections to work with parallelism. The main sequential collection is integrated with parallel ones. Let's see some classes: ParArray.


1 Answers

See "A Generic Parallel Collection Framework", the paper by Martin Odersky et al that discusses the new parallel collections. Page 8 has a section "Parallel Views" that talks about how view and par can be used together, and how this can give the performance benefits of both views and parallel computation.

As for your specific example, that is definately a bug. The exists method also breaks, and having it break on one list breaks it for all other lists, so I think it is a problem where operations that may be aborted part way through (find and exists can stop once the have an answer) manage to break the thread pool in some way. It could be related to the bug with exceptions being thrown inside functions passed to parallel collections. If so, it should be fixed in 2.10.

like image 159
wingedsubmariner Avatar answered Sep 28 '22 12:09

wingedsubmariner