Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Mapreduce job in hue steps not clear

I'm trying run mapreduce job in hue. I followed the following steps:

  Fill out the Edit Node form
    1)Enter ooziemrwf as the name, and "oozie MapReduce workflow" as the description.

   2) Navigate to the JAR file location and upload the JAR file.

   3) Click the Add Property button and add each of these four properties: 
    Property Name
    Value
    mapred.mapper.class org.apache.oozie.example.SampleMapper
    mapred.reducer.class    org.apache.oozie.example.SampleReducer
    mapred.output.dir   /user/mapr/mapreducejob/wfoutput
    mapred.input.dir    /oozie/examples/input-data/text

   4) Navigate to the JAR file and select it as the input path.

    5)Click Done. The MapReduce action appears in the workflow.

   6) Click Save.

Under Actions in the navigation panel, click Submit. When the Submit this job? dialog appears, click Submit.

I uploaded the working jar added properties and I'm not able understand the 4th step i.e.Navigate to the JAR file and select it as the input path. Can some one please tell what is that exactly? you can refer http://doc.mapr.com/display/MapR/Working+with+Hue

like image 576
jane Avatar asked Sep 16 '25 05:09

jane


1 Answers

Not sure if you are able to "Run" Anything at this point. There are a number of things that can be going wrong.

I am assuming you are using HUE to create the Job Def.

  1. Check that the map-reduce runner is able to copy the directories on your hadoop cluster. This usually means you have a /var/tmp/oozie directory in HDFS (with correct user/group permissions). If the directory is missing there will be issues.
  2. if you are using MapReduce2 vs mapred, there may be issues with Yarn trying to commingle mapred and mapreduce packages. in that case the best bet is to run as a "java" job from HUE, then you can explicitly manage weather it is MapReduce or mapred.
  3. Include some input and some output from your run. Showing your job def, and output errors would be helpful.
like image 77
Q Boiler Avatar answered Sep 19 '25 15:09

Q Boiler