Host and port to use to list a directory in hdfs

Question

First of all, I'm using HortonWorks Sandbox as Hadoop dist, with no custom configuration at all.

Once connected on the sandbox, I'm able to list files of a HDFS directory doing :

[root@sandbox ~]# hadoop fs -ls hdfs:///user/guest

but if I try to specify a host and port I get only errors :

[root@sandbox ~]# hadoop fs -ls hdfs://localhost:8020/user/guest ls: Call From sandbox.hortonworks.com/10.0.2.15 to localhost:8020 failed on connection exception: java.net.ConnectException: Connexion refusée; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

[root@sandbox ~]# hadoop fs -ls hdfs://localhost:9000/user/guest ls: Call From sandbox.hortonworks.com/10.0.2.15 to localhost:9000 failed on connection exception: java.net.ConnectException: Connexion refusée; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

Once I know the correct host and port to use, I'll be able to use them in my Java call :

Path pt = new Path("hdfs://host:port/user/guest/test-text-file.txt");

Answer 1

Check the value of property fs.defaultFS in core-site.xml this contains the ip-address/hostname and port on which NameNode daemon should bind to when it start's up.

I see that you are using hortonworks sandbox, here is the property in core-site.xml and its located in /etc/hadoop/conf/core-site.xml

<property>
  <name>fs.defaultFS</name>
  <value>hdfs://sandbox.hortonworks.com:8020</value>
</property>

So, you could try something like this:

hadoop fs -ls hdfs://sandbox.hortonworks.com:8020/user/guest 

Or you could also replace the ip address of sandbox.hortonworks.com from its respective entry in /etc/hosts, on my vm which looks something like this:

127.0.0.1       localhost.localdomain localhost
192.168.1.3 sandbox.hortonworks.com sandbox

So, I could try this as well:

hadoop fs -ls hdfs://192.168.1.3:8020/user/guest