Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Hadoop Resource Manager not starting

I am new to Hadoop so I started setting up partially distributed Hadoop cluster on localhost. I created a wordcount program jar to run but when I run this jar the following error is shown for nine times:

NFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9040. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

Also when I type jps command my services started are:

9154 DataNode

9777 NodeManager

8967 NameNode

9441 SecondaryNameNode

10060 Jps

core-site.xml

<configuration>

  <property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:9000</value>
  </property>

  <property>
  <name>hadoop.tmp.dir</name>
  <value>/usr/local/hadoop/tmp</value>
  <description>A base for other temporary directories.</description>
  </property>

  <property>
     <name>fs.file.impl</name>
     <value>org.apache.hadoop.fs.LocalFileSystem</value>
     <description>The FileSystem for file: uris.</description>
  </property>
  -
  <property>
     <name>fs.hdfs.impl</name>
     <value>org.apache.hadoop.hdfs.DistributedFileSystem</value>
     <description>The FileSystem for hdfs: uris.</description>
  </property>

</configuration>

yarn-site.xml

<configuration>

  <!-- Site specific YARN configuration properties -->
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
  <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>localhost:9025</value>
  </property>
  <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>localhost:9030</value>
  </property>
  <property>
    <name>yarn.resourcemanager.address</name>
    <value>localhost:9040</value>
  </property>
  <property>
    <name>yarn.nodemanager.address</name>
    <value>localhost:9050</value>
  </property>
  <property>
    <name>yarn.nodemanager.localizer.address</name>
    <value>localhost:9060</value>
  </property>

</configuration> 

mapred-site.xml

<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>

  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:8021</value>
  </property>

</configuration>

Help appreciated. How do I resolve this ??

like image 432
Shahadeo Avatar asked Feb 14 '15 07:02

Shahadeo


People also ask

How do I check if resource manager is running or not in Hadoop?

This can be achieved using the Hadoop Resource Manager Uptime test. This test continuously tracks and reports the uptime of the resource manager, thus revealing whether/not a scheduled reboot occurred.

What if resource manager is down?

Failure of the resource manager is serious because, without it, neither jobs nor task containers can be launched. In the default configuration, the resource manager is a single point of failure, since, in the (unlikely) event of machine failure, all running jobs fail—and can't be recovered.


1 Answers

The most probable reason for no ResourceManager daemon would be an error in HTTP port binding. Go through the log files of yarn-resourcemanager-*. You can easily find the cause of error.

like image 122
pbskumar Avatar answered Oct 14 '22 01:10

pbskumar