Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Scala class inheriting from a Java generic class

I am trying to write a Hadoop mapper class in Scala. As a starting point, I have taken a Java example from the book "Hadoop: the Definitive Guide" and tried to port it to Scala.

The original Java class extends org.apache.hadoop.mapreduce.Mapper:

public class MaxTemperatureMapper
    extends Mapper<LongWritable, Text, Text, IntWritable>

and overrides the method

public void map(LongWritable key, Text value, Context context)
    throws IOException, InterruptedException

This methods gets called and works properly (I tested using a unit test and then run it with yarn).

My attempt at a Scala port is:

class MaxTemperatureMapperS extends Mapper[LongWritable, Text, Text, IntWritable]

and then the method

@throws(classOf[IOException])
@throws(classOf[InterruptedException])
override def map(key: LongWritable, value: Text, context: Context): Unit = 
{
    ...
}

but the Scala compiler issues an error:

error: method map overrides nothing.

So I thought the two methods had the same signature in Scala and Java, but apparently I am missing something. Can you give me some hint?

like image 207
Giorgio Avatar asked Feb 08 '18 18:02

Giorgio


1 Answers

Sometimes the best way to do this is by letting your IDE work for you:

class Test extends Mapper[LongWritable, Text, Text, IntWritable] {
  override def map(key: LongWritable, value: Text, context: Mapper[LongWritable, Text, Text, IntWritable]#Context): Unit = ???
}

In this case the problem is that the definition of the class Context "lives" inside the class Mapper so you need to use the # syntax

like image 195
Mikel San Vicente Avatar answered Sep 19 '22 11:09

Mikel San Vicente