How to bypass or satisfy scala generic types for Thrift classes

Question

I am trying to write a generic scala utility function for dealing with Apache Thrift generated Java classes. All Thrift generated Java classes extend the TBase interface with the following signature:

public interface TBase<T extends TBase<?,?>, F extends TFieldIdEnum>

This definitely has always proved to be problematic when making generic thrift utility functions and in the Java world I have typically solved this by just using @SuppressWarnings("rawtypes") knowing that I was dealing with it correctly and effectively ignoring the compiler telling me it can't verify the matching types.

In the scala world, I don't seem to have the luxury or ignoring these types and have run into a situation where I can't seem to figure out how to satisfy the generic type requirements.

So I have created my own function which converts the given non-thrift object to an instance of the requested Thrift class with a signature similar to:

def myFunc[T <: TBase[T, E], E<: TFieldIdEnum](obj: Any, clazz: Class[T]): T = {

So my issue here is that to call this function I have to do something like:

myFunc[MyThriftClass,MyThriftClass._Fields]( obj, classOf[MyThriftClass] )

Without the second type parameter explicitly specified, the compilation fails with:

error: inferred type arguments [MyThriftClass,Nothing] do not conform to method myFunc's type parameter bounds [T <: org.apache.thrift.TBase[T,E],E <: org.apache.thrift.TFieldIdEnum]
    myFunc( null, classOf[MyThriftClass])
    ^
error: type mismatch;
found   : Class[MyThriftClass](classOf[MyThriftClass])
required: Class[T]
   myFunc( null, classOf[MyThriftClass])

When I pass both type parameters, everything works though I would prefer to only need to pass the class.

But what I can't seem to figure out how to get this to work if I don't know the types at compile time.

For example, thrift metadata is defined via FieldValueMetaData objects and when there's a sub-struct, the provided metadata object is StructMetaData which contains a field

public final Class<? extends TBase> structClass;

Given that this class is missing the type parameters, I don't know how to call my method with it while satisfying the generic type constraints. In the case of recursion I managed to bypass this by casting the substruct class to an instance of Class[T] which is not true because it's the parent type not the sub-type but due to type erasure it makes the compiler happy and works at runtime. But if I don't already have a generic type to cast to I don't know how to handle this aside from maybe a really hacky approach where I cast it to some fake shell of a TBase class just to make the compiler happy.

Is there a proper way to set this up to make the compiler happy with the real types? Alternatively is there a way to bypass this similarly to how I would in pure Java with @SuppressWarnings("rawtypes"). Otherwise if not then maybe I'll just have to port this logic over to Java and call it from Scala to make everything happy. Before doing this though I'd either like to learn what I'm missing or learn why what I want is not possible.

Edit:

Really what I'm looking for is some way in scala to call my existing myFunc method given a Tbase[_,_]. In Java I'd do this with suppressing raw types but in scala I'd like to either know a way to satisfy the types and allow my method to be called or a way to suppress them and allow my method to be called.

Edit 2:

Figured out a way to do what I want but still looking for cleaner solutions if they exist. What I ended up with is the following:

def myFunc[T <: TBase[_,_],T2 <: TBase[T2, E2], E2<: TFieldIdEnum](obj: Any, clazz: Class[T]): T = {
  _myFunc[T2,E2]( obj, clazz.asInstanceOf[Class[T2]] ).asInstanceOf[T]
}

def _myFunc[T <: TBase[T, E], E<: TFieldIdEnum](obj: Any, clazz: Class[T]): T = {

To be honest I don't know why it works because I haven't specified what types T2 or E2 are but this does seem to make it compile and run.

Answer 1

Short answer, that possibly can fit your purpose.

This statement is not correct.

In the scala world, I don't seem to have the luxury of ignoring these types

In the scala world, you can erase generic type parameters using forSome:

def myFunc[T <: TBase[U, E] forSome {type U; type E}, E<: TFieldIdEnum](obj: Any, clazz: Class[T])

construct T <: TBase[U, E] forSome {type U; type E} states that T is a subtype of a TBase with certain but not important type parameters.
you can shorten forSome to just _:

def myFunc[T <: TBase[_, _], E<: TFieldIdEnum](obj: Any, clazz: Class[T])

optionally, you can rid of parameter clazz by summoning implicit class tag:

def myFunc[T <: TBase[_, _]: ClassTag, E<: TFieldIdEnum](obj: Any) = {
  val c = implicitly[ClassTag[T]].runtimeClass

Much longer answer with a solution without dropping generic parameters.

In this piece of code:

def myFunc[T <: TBase[T, E], E<: TFieldIdEnum](obj: Any, clazz: Class[T]): T = ???

it seems that you trying to do kind of "type parameter currying", and then type parameter inference. The quirk is that scala does not have by-default type parameter currying in the way, that some of type parameters could be omitted and inferred by scalac. In scala you should explicitly set type parameters you want to set, and type parameters you want to be inferred from usage. This is done using auxiliary class and its companion object. Having

  def foo_1[G[_], A](fb: G[A]):G[A] = ???

Which is called:

  val v: List[Int] = ???
  foo_1[List, Int](v)

Because, you know, higher kinds cannot be that easily inferred in some situations. Then we introduce special empty case class foo[G[_]]() which is used to carry this type. We could create its instance via the apply method to have nice syntax.

  object foo{
    def apply[G[_]]: foo[G] = new foo[G]()
  }

Then we could put another apply method which will introduce second type parameter B

  case class foo[G[_]](){
    def apply[A](fa: G[A]): G[A] = foo_1[G,A](fa)
  }

Than we could have conciese boilerplateless syntax:

  val v: List[Int] = ???

  val v1 = foo_1[List, Int](v)
  val v2 = foo[List](v)
} 

Note that foo[List](v) and foo.apply[List].apply[Int](v) are just the same because of apply related syntax sugar.

The place where this method not works is where the type parameter B could not be inferred by the compiler. I hope your case is not that case.

Last, thus not least, as you haven't asked for this, but, you actually can avoid passing clazz: Class[T] explicitly as a function parameter, but instead you can turn it to an implicit parameter:

def myFunc[T <: TBase[T, E], E<: TFieldIdEnum](obj: Any)(implicit ct: ClassTag[T]): T = ???

and access it as regular variable from your function. To wrap everything up, you must pass this class tag through auxiliary class. Also, to use the trick you should erase the dependency of T on type E to allow using that via forSome.

  trait F[A, B]
  trait U

  object foo {
    def apply[T <: F[T, E] forSome {type E}]: foo[T] = new foo[T]
  }

And, to restore type E, you can ask compiler to infer such parameter E that both undegoes requirements: E <: U, and T <: F[T,E] via implicit evidence of type <:<, which generated by compiler.

  case class foo[T <: F[T, E] forSome {type E}]() {
    def apply[E <: U](a: Any)(implicit e: <:<[T, F[T, E]], ct: ClassTag[T]): T = {
      ???
      //here goes your code.
    }
  }

I hope you can use the fact T <: F[T,E] in a form of an evidence(which also subtype of T => F[T,E]).

This is how the syntax should be used:

  trait E extends U
  trait T extends F[T,E]

  foo[T].apply("abc")

Answer 2

I would recommend using scrooge to generate your thrift classes.

It generates native scala code which is a lot nicer and more idiomatic to deal with scala, and dancing around the (mutable!!!) java classes.

Scrooge-generated structures extend ThriftStruct which is not parametrized, and solves all your problem in on shot.