Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

java.io.IOException: Initialization of all the collectors failed. Error in last collector was :null

I am a newbie in MapReduce and I am trying to find a solution to a problem.I am trying to chain two map reduce jobs.The first job is getting executed but on the second job I am getting an error as follows

     INFO mapreduce.Job: Task Id : attempt_1445271708293_0055_m_000000_1, Status : FAILED
Error: java.io.IOException: Initialization of all the collectors failed.              Error in last collector was :null
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:414)
    at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:698)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1011)
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:402)
    ... 9 more
like image 701
vin Avatar asked Oct 20 '15 01:10

vin


2 Answers

I had imported Text in Driver class as

import com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider.Text

instead of

import org.apache.hadoop.io.Text

because of which I was getting the error, once I rectified the mistake it started working fine.

like image 61
Arun Avatar answered Sep 21 '22 13:09

Arun


I have got the same error when output types from map task didn't matched with input types of reduce task.

Last two arguments in mapper should have the same type as first two in reducer.

public class ByteCalculationMapper extends Mapper<LongWritable, Text, Text, IntWritable> {

public class ByteCalculationReducer extends Reducer<Text, IntWritable, Text, Text> {
like image 33
Max Pavlov Avatar answered Sep 25 '22 13:09

Max Pavlov