Hadoop dfs -ls returns list of files in my hadoop/ dir

Question

I've set up a sigle-node Hadoop configuration running via cygwin under Win7. After starting Hadoop by

bin/start-all.sh

I run

bin/hadoop dfs -ls

which returns me a list of files in my hadoop directory. Then I run

bin/hadoop datanode -format
bin/hadoop namenode -format

but -ls still returns me the contents of my hadoop directory. As far as I understand it should return nothing(empty folder). What am I doing wrong?

Answer 1

Did you edit the core-site.xml and mapred-site.xml under conf folder ? It seems like your hadoop cluster is in local mode.

Answer 2

I know this question is quite old, but directory structure in Hadoop has changed a bit (version 2.5 ) Jeroen's current version would be.

hdfs dfs -ls hdfs://localhost:9000/users/smalldata

Also Just for information - use of start-all.sh and stop-all.sh has been deprecated, instead one should use start-dfs.sh and start-yarn.sh

Answer 3

I had the same problem and solved it by explicitly specifying the URL to the NameNode.

To list all directories in the root of your hdfs space do the following:

./bin/hadoop dfs -ls hdfs://<ip-of-your-server>:9000/

The documentation says something about a default hdfs point in the configuration, but I cannot find it. If someone knows what they mean please enlighten us.

This is where I got the info: http://hadoop.apache.org/common/docs/r0.20.0/hdfs_shell.html#Overview

Answer 4

Or you could just do:

• Run stop-all.sh.
• Remove dfs data and name directories
• Namenode -format
• Run start-all.sh