How to reference subclasses of static Java classes with generics in Scala

Question

I have this Java code:

public class TestMapper extends AppEngineMapper<Key, Entity, NullWritable, NullWritable> {
  public TestMapper() {
  }
// [... other overriden methods ...]
      @Override
      public void setup(Context context) {
        log.warning("Doing per-worker setup");
      }
}

...which I've converted to:

class TestMapper extends AppEngineMapper[Key, Entity, NullWritable, NullWritable] {
// [... other overriden methods ...]
      override def setup(context: Context) {
        log.warning("Doing per-worker setup")
      }
}

Now the actual problem:

Context is defined as a nested class within the org.apache.hadoop.mapreduce.Mapper class:

        public static class Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT>   {
    //[... some other methods ...]
protected void setup(org.apache.hadoop.mapreduce.Mapper<KEYIN,VALUEIN,KEYOUT,VALUEOUT>.Context context) throws java.io.IOException, java.lang.InterruptedException { /* compiled code */ }
        public class Context extends org.apache.hadoop.mapreduce.MapContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT> {

        public Context(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.mapreduce.TaskAttemptID conf, org.apache.hadoop.mapreduce.RecordReader<KEYIN,VALUEIN> taskid, org.apache.hadoop.mapreduce.RecordWriter<KEYOUT,VALUEOUT> reader, org.apache.hadoop.mapreduce.OutputCommitter writer, org.apache.hadoop.mapreduce.StatusReporter committer, org.apache.hadoop.mapreduce.InputSplit reporter) throws java.io.IOException, java.lang.InterruptedException { /* compiled code */ }

        }

So I can't tell my Scala class where/what Context actually is. If Mapper had no generics I could reference Context via

Mapper#Context

but how can I tell that Mapper has Generics?

Mapper[_,_,_,_]#Context

...didn't work.

Answer 1

You have to supply the exact base type for your type projection, in your case

Mapper[Key, Entity, NullWritable, NullWritable]#Context

so overriding setup would be written as

override def setup(context: Mapper[Key, Entity, NullWritable, NullWritable]#Context)

Usage can be simplified by introducing a type alias

class TestMapper extends AppEngineMapper[Key, Entity, NullWritable, NullWritable] {

  type Context = Mapper[Key, Entity, NullWritable, NullWritable]#Context

  override def setup(context: Context) = {
      // ...
   }
}

If you want to write multiple mappers you can refactor this into a trait that can be mixed into your implementation:

trait SMapper[A,B,C,D] extends Mapper[A,B,C,D] {
  type Context = Mapper[A,B,C,D]#Context
}

class TestMapper extends AppEngineMapper[Key, Entity, NullWritable, NullWritable]
                    with SMapper[Key, Entity, NullWritable, NullWritable] {
  override def setup(context: Context) = {
     // ...
  }
}

or for plain hadoop:

class TestMapper extends SMapper[Key, Entity, NullWritable, NullWritable] {
  override def setup(context: Context) = {
     // ...
  }
}