HADOOP / YARN - Are the ResourceManager and the hdfs NameNode always installed on the same host?

Question

Are the “resource manager” and the “hdfs namenode” always installed on the same host?

1) When I want to send an http request (YARN REST API) to get new application id I am using this web uri:

http://<rm http address:port>/ws/v1/cluster/apps/new-application

This port is the Resource-Manager Web UI http port, it’s default value is 8088 as shown in img1: source for img1

2) When I want to send an http request (WebHDFS REST API) for hdfs commands, for example to get file status I am using this web uri:

http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETFILESTATUS

This <PORT> is the NameNode WebUI http Port, it’s default value is 50070 as shown in img2: source for img2

Are the hosts of those components (ResourceManager and NaneNode) are always installed on the same host?

Any help would be appreciated, Thanks!

Answer 1

It is not necessary to run resourceManager and nameNode on a single machines, we have such configuration when working in a single node/small cluster---with few nodes---configuration. When there is large cluster, master nodes usually run resourceManager, nameNode and secondaryNamenode on different machines.

Have a look at these links:

Master Nodes in Hadoop Clusters

HortonWorks: Typical Hadoop Cluster