Reducer starts before mapper has finished

Question

I am running a Map Reduce Program. However I am getting similar output even though I am running it with only mapper or both with mapper and reducer.

After this it never completes.It hangs up there on.

I am not getting why reducer is getting started before mapper has finished 100%? What might be potential problems?

Output:

Map 10% Reduce 0%
Map 19% Reduce 0%
Map 21% Reduce 0%
Map 39% Reduce 0%
Map 49% Reduce 0%
Map 63% Reduce 0% 
Map 67% Reduce 0% 
Map 68% Reduce 0% 
Map 68% Reduce 22%
Map 69% Reduce 22%

Here is a mapper code:

public class EntityCountMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
  static String total_record="";

  @Override
  protected void map(LongWritable baseAddress, Text line, Context context)
        throws IOException, InterruptedException {

    Text entity=new Text();
    IntWritable one=new IntWritable(1);

    total_record=total_record.concat(line.toString());
    String[] fields=total_record.split("::");
    if(fields.length==24)
    {
        entity.set(fields[22].trim());          
        context.write(entity,one);
        total_record="";
    }       
  }
}

Answer 1

The reduce phase has 3 steps: shuffle, sort, reduce. Shuffle is where the data is collected by the reducer from each mapper. This can happen while mappers are generating data since it is only a data transfer. On the other hand, sort and reduce can only start once all the mappers are done. You can tell which one MapReduce is doing by looking at the reducer completion percentage: 0-33% means its doing shuffle, 34-66% is sort, 67%-100% is reduce. This is why your reducers will sometimes seem "stuck" at 33%-- it's waiting for mappers to finish.