Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why 'mapred-site.xml' is not included in the latest Hadoop 2.2.0?

Latest build of Hadoop provides mapred-site.xml.template Do we need to create a new mapred-site.xml file using this? Any link on documentation or explanation related to Hadoop 2.2.0 will be much appreciated.

like image 433
pcdhan Avatar asked Nov 23 '13 17:11

pcdhan


1 Answers

I believe it's still required. For our basic Hadoop 2.2.0 2-node cluster setup that we have working I did the following from the setup documentation.

"

From the base of the Hadoop installation, edit the etc/hadoop/mapred-site.xml file. A new configuration option for Hadoop 2 is the capability to specify a framework name for MapReduce, setting the mapreduce.framework.name property. In this install we will use the value of "yarn" to tell MapReduce that it will run as a YARN application. First, copy the template file to the mapred-site.xml.

cp mapred-site.xml.template mapred-site.xml

Next, copy the following into Hadoop etc/hadoop/mapred-site.xml file and remove the original empty tags.

 <configuration> 
 <property>   
 <name>mapreduce.framework.name</name>  
 <value>yarn</value>  
 </property>   
 </configuration>

"

Wrt documentation, I found this the most useful. Also, etc/hosts configs for cluster setup and other cluster related configs were a bit hard to figure out.

like image 114
Vishal Avatar answered Oct 11 '22 10:10

Vishal