Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Hadoop 2.x -- how to configure secondary namenode?

Tags:

hadoop

hdfs

I have an old Hadoop install that I'm looking to update to Hadoop 2. In the old setup, I have a $HADOOP_HOME/conf/masters file that specifies the secondary namenode.

Looking through the Hadoop 2 documentation I can't find any mention of a "masters" file, or how to setup a secondary namenode.

Any help in the right direction would be appreciated.

like image 204
codecraig Avatar asked Jun 19 '14 10:06

codecraig


2 Answers

The slaves and masters files in the conf folder are only used by some scripts in the bin folder like start-mapred.sh, start-dfs.sh and start-all.sh scripts.

These scripts are a mere convenience so that you can run them from a single node to ssh into each master / slave node and start the desired hadoop service daemons.

You only need these files on the name node machine if you intend to launch your cluster from this single node (using password-less ssh).

Alternatively, You can also start an Hadoop daemon manually on a machine via

bin/hadoop-daemon.sh start [namenode | secondarynamenode | datanode | jobtracker | tasktracker]

In order to run the secondary name node, use the above script on the designated machines providing the 'secondarynamenode' value to the script

like image 165
Vijay Bhoomireddy Avatar answered Oct 20 '22 20:10

Vijay Bhoomireddy


See @pwnz0r 's 2nd comment on answer on How separate hadoop secondary namenode from primary namenode?

To reiterate here:

In hdfs-site.xml:
<property> 
   <name>dfs.secondary.http.address</name>
   <value>$secondarynamenode.full.hostname:50090</value>
   <description>SecondaryNameNodeHostname</description>
</property>
like image 36
Will Dennis Avatar answered Oct 20 '22 20:10

Will Dennis