Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to print on console during MapReduce job execution in hadoop

I want to print each step of my "map" after its execution on the console.

Something like

System.out.println("Completed Step one"); System.out.println("Completed Step two");

and so on

Is there a special command to do that or is it not possible at all, as System.out.println doesn't seem to work at all ?

Please guide

like image 504
Shrish Bajpai Avatar asked Aug 04 '11 13:08

Shrish Bajpai


People also ask

Where is the output of Map tasks stored in MapReduce?

A MapReduce job usually splits the input data-set into independent chunks which are processed by the map tasks in a completely parallel manner. The framework sorts the outputs of the maps, which are then input to the reduce tasks. Typically both the input and the output of the job are stored in a file-system.

What is input and output of MapReduce?

They are sequenced one after the other. The Map function takes input from the disk as <key,value> pairs, processes them, and produces another set of intermediate <key,value> pairs as output. The Reduce function also takes inputs as <key,value> pairs, and produces <key,value> pairs as output.


1 Answers

You can use System.out.println(... To see whats been printed, open the jobtracker web console. Then navigate to your submitted job's link. On the Job page, you will see links to all your individual Map/Reduce tasks. Follow the links.. you will find a link that displays the dump of both the Standard output as well as the Standard error.

like image 57
arun_suresh Avatar answered Oct 21 '22 10:10

arun_suresh