Generic type parameters for conversion to parallel collection

Question

I have this function to convert an Array to a ParArray, giving the number of threads as parameter:

def parN[T](collection: Array[T], n: Int) = {
    val parCollection = collection.par
    parCollection.tasksupport = new ForkJoinTaskSupport(
      new concurrent.forkjoin.ForkJoinPool(n))
    parCollection
}

Now I'd like to make this generic, such that it works with collections other than Array:

def parN[S, T[S] <: Parallelizable[S, ParIterable[S]]](collection: T[S], n: Int) = {
    val parCollection = collection.par
    parCollection.tasksupport = new ForkJoinTaskSupport(
      new concurrent.forkjoin.ForkJoinPool(n))
    parCollection
}

But when I call it with parN(Array(1, 2, 3), 2), I get this error:

inferred type arguments [Int,Array] do not
conform to method parN's type parameter bounds
[S,T[S] <: scala.collection.Parallelizable[S,scala.collection.parallel.ParIterable[S]]]

On the other hand, this is working:

val x: Parallelizable[Int, ParIterable[Int]] = Array(1, 2, 3)

Any ideas what might be wrong with my type parameters?

Answer 1

First of all, note that this is a problem specific to Array: your method works for List or any other normal collection (except those with multiple type parameters, or none). Example:

scala> parN(List(1,2,3), 2)
res17: scala.collection.parallel.ParIterable[Int] = ParVector(1, 2, 3)

scala> parN(Set(1,2,3), 2)
res18: scala.collection.parallel.ParIterable[Int] = ParSet(1, 2, 3)

Array is always a special case when it comes to collections, because... it is not a collection. Because of Java, its definition is final class Array[T] extends Serializable with Cloneable. However, there exist two implicits, available everywhere, that can convert Array to a collection type (ArrayOps and WrappedArray). And these types do implement Parallelizable, so everything should be fine... except type inference gets in the way:

Your type parameter is only defined as T[S], so when receiving an Array[Int], it will gladly infer Array[Int], and then check the bounds: failure, Array doesn't extend Parallelizable. Game over.

I see two options:

• You can explicitly say that you want a Parallelizable:

  def parN[S](collection: Parallelizable[S, ParIterable[S]], n: Int)
  

  Or if you need access to the actual type T (in your case no, but who knows):

  def parN[S, T[S] <: Parallelizable[S, ParIterable[S]]](collection: T[S] with
     Parallelizable[S, ParIterable[S]], n: Int)
  

• or you can accept anything that can be implicitly converted to a Parallelizable, using an implicit parameter:

  def parN[S, T <% Parallelizable[S, ParIterable[S]]](collection: T, n: Int)
  

  which is the short version of:

  def parN[S, T](collection: T, n: Int)(implicit ev: T => 
    Parallelizable[S, ParIterable[S]])
  

All of those should work. In your case I would recommend the very first one: it is the most readable one that does the job.