How do I solve this Scala function parameter type erasure error?

Question

I am creating a map-reduce framework, and right now I am trying to create a builder class to instantiate the processing pipeline. This builder needs to hold a list of functions specified by the user, so later it can construct the pipeline using these functions as arguments to instantiate Worker objects.

I am using case classes to create function "holders", and also for the "workers". My problem is that when I go on to analyse the holders, if I pattern-match on them the type information from the function seems to be missing due to type erasure. Here is some minimal code that appears to reproduce the problem I am having.

trait Holders

case class MapHolder[A, B](f: A => B) extends Holders

trait Worker

case class MapWrk[A, B](f: A => B) extends Worker

object MyTypeErasureProblem extends App {

  val myFunc = MapHolder((x: Int) => x + 10)

  def buildWorker(hh: Holders) =
    hh match {
      case MapHolder(f) => MapWrk(f)
    }

  println(buildWorker(myFunc).f(10))
}

The compiler error is

Error:(22, 35) type mismatch;
 found   : Nothing => Any
 required: A => Any
      case MapHolder(f) => MapWrk(f)
                                  ^
Error:(26, 33) type mismatch;
 found   : Int(10)
 required: Nothing
  println(buildWorker(myFunc).f(10))
                                ^

The problem can be solved if we define buildWorker like this:

def buildWorker[A,B](hh: MapHolder[A,B]) = ...

But I need to process different kinds of Holders, including

case class ReduceHolder[V](f: (V,V) => V) extends Holders

which, by the way, works just fine in a similar code, with no errors or warnings at all. In fact, only the generic type A=>B seems to be a problem, because the function argument becomes Nothing, passing around objects with other generic types work, e.g. Tuple2[A,B].

This looks like a problem related to type erasure to me, but how can I go around it? I have tried to ClassTag all the things, but it didn't work.

MORE INFO Here's an update with the approach suggested by Travis, using a buildWorker method inside the Holder.

This here works the way I needed:

case class DataHolder[T](f: T) {
  def buildWorker() = DataWrk[T](f)
}

case class DataWrk[T](f: T)

object MyTypeErasureProblem2 extends App {
  val pipeline = List(
    DataHolder[Int](10).buildWorker(),
    DataHolder[String]("abc").buildWorker()
  )
  val result = pipeline collect {
    case DataWrk(f: Int) => "Int data"
    case DataWrk(f: String) => "String data"
  }
  result foreach println
}

Output:

Int data
String data

But if we use function types, it doesn't work anymore:

case class MapHolder[T](f: T => T) {
  def buildWorker() = MapWrk[T](f)
}

case class MapWrk[T](f: T => T)

object MyTypeErasureProblem extends App {
  val pipeline = List(
    MapHolder[Int]((x: Int) => x + 10).buildWorker(),
    MapHolder[String]((x: String) => x + "abc").buildWorker()
  )
  val result = pipeline collect {
    case MapWrk(f: (Int => Int)) => "Int endofunction"
    case MapWrk(f: (String => String)) => "String endofunction"
  }
  result foreach println
}

Output:

Int endofunction
Int endofunction

When we try to match on the instantiated Workers, the first case always matches.

Answer 1

You could add little bit type clarity to your buildWorker definition:

  def buildWorker[A, B](hh: Holders) =
    hh match {
      case MapHolder(f: (A => B)) => MapWrk(f)
    }

UPDATE: Here few more approaches to make type-less buildWorker method: First to use existential types, but case holder:(MapHolder[A, B] forSome {type A; type B}) did not compile for some reason, so one attempt could be:

  type MapHolderGeneric = MapHolder[A, B] forSome {type A; type B}

  def buildWorker(hh: Holders):Worker = 
    hh match {
      case holder: MapHolderGeneric => MapWrk(holder.f)
    }

next is fairly equivalent:

  case holder: MapHolder[_,_] => MapWrk(holder.f)

Both know Nothing about types. So their A and B would be inferred to Nothing in most cases as Travis pointed and this is just a way to trick the compiler.

Your example code

   case DataWrk(f: Int) => "Int data"
   case DataWrk(f: String) => "String data"

could never work because types are not reified in DataWrk instances, so at runtime it can'be checked. But we could reify them by hand using scala-reflect:

import scala.reflect.runtime.universe._

trait Holders

case class MapHolder[T](f: T => T)(implicit tag: TypeTag[T]) {
  def buildWorker() = MapWrk[T](f)
}

trait Worker

class MapWrk[T](val f: T => T)(implicit val tag: TypeTag[T]) extends Worker

object MapWrk {

  def apply[T](f: T => T)(implicit tag: TypeTag[T]) = new MapWrk[T](f)

  def unapply(worker: Worker): Option[(_ => _, Type)] = worker match {
    case mapWrk: MapWrk[_] => Some((mapWrk.f, mapWrk.tag.tpe))
    case _ => None
  }
}

object MyTypeErasureProblem extends App {
  val pipeline = List(
    MapHolder[Int]((x: Int) => x + 10).buildWorker(),
    MapHolder[String]((x: String) => x + "abc").buildWorker()
  )

  val result = pipeline collect {
    case MapWrk(f, tpe) if tpe =:= typeOf[Int] => "Int endofunction"
    case MapWrk(f, tpe) if tpe =:= typeOf[String] => "String endofunction"
  }

  result foreach println
}

this is printing out what you've expected

more straightforward solution was just to send some implicit type to unapply method, and just use type parameter to specify what type we are matching here, but this is not implemented yet, so i assume f inside the pattern maching could not be typed yet. But it would probably in 2.12