Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

hadoop datanode unable to start. "does not contain a valid host:port authority"

Tags:

xml

hadoop

I'm currently using hadoop 1.2.1 (because I need to run a spatial processing software only support this version). I'm trying to deploy in multinode mode with one master and three slaves.

I'm sure I'm able to ssh between all master and slaves without password (including themselves). Also the hostname on each node is correct. Each node shares the same host file:

192.168.56.101 master
192.168.56.102 slave1
192.168.56.103 slave2
192.168.56.104 slave3

I keep having problems in the slaves node, error log info is as follows,

2015-05-21 23:39:16,841 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: file:///
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:212)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:244)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:236)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:359)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:321)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1712)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1651)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1669)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1795)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:181

Configurations in core-site.xml

<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://master:9000</value>
    </property>
</configuration>

In mapred-site.xml:

<configuration>
    <property>
        <name>mapred.job.tracter</name>
        <value>master:8012</value>  
    </property>
</configuration>

In hdfs-site.xml:

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>3</value>
    </property>
</configuration
like image 395
xuanyue Avatar asked May 22 '15 07:05

xuanyue


2 Answers

There could be a problem with the naming convention of your node hostnames. Make sure they do not contain symbols like "_". Check Wikipedia for restrictions.

like image 140
RiteshC Avatar answered Nov 03 '22 00:11

RiteshC


Try to change the "master" to the actual ip address, in all your config files.

like image 32
gwgyk Avatar answered Nov 03 '22 02:11

gwgyk