I am a newbie in MapReduce and I am trying to find a solution to a problem.I am trying to chain two map reduce jobs.The first job is getting executed but on the second job I am getting an error as follows
INFO mapreduce.Job: Task Id : attempt_1445271708293_0055_m_000000_1, Status : FAILED
Error: java.io.IOException: Initialization of all the collectors failed. Error in last collector was :null
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:414)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:698)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1011)
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:402)
... 9 more
I had imported Text in Driver class as
import com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider.Text
instead of
import org.apache.hadoop.io.Text
because of which I was getting the error, once I rectified the mistake it started working fine.
I have got the same error when output types from map task didn't matched with input types of reduce task.
Last two arguments in mapper should have the same type as first two in reducer.
public class ByteCalculationMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
public class ByteCalculationReducer extends Reducer<Text, IntWritable, Text, Text> {
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With