Menu
I'm new to hadoop. I am trying to create a directory in hdfs but I am not able to create.
I have logged into "hduser" hence I assumed /home/hduser" pre-exists as Unix fs. So I tried to create hadoop directory using below command.
[hduser@Virus ~]$ hadoop fs -mkdir /home/hduser/mydata/
14/12/03 15:04:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
mkdir: `/home/hduser/mydata/': No such file or directory
After online search, i thought of it is possible that hadoop is not able to understand "/home/hduser" or as I m using hadoop2 where mkdir wont work like Unix command "madir -p" (recursively). Hence I tried to create "/mydata" but no luck.
[hduser@Virus ~]$ hadoop fs -mkdir /mydata
14/12/03 15:09:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
mkdir: Cannot create directory /mydata. Name node is in safe mode.
I tried to leave the safemode but still issue persists.
[hduser@Virus ~]$ hdfs dfsadmin -safemode leave
14/12/03 15:09:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Safe mode is OFF
I also tried with "/user/mydata" as "/user" is the directory which hadoop took as home.
[hduser@Virus ~]$ hadoop fs -mkdir /user/mydata
14/12/03 15:36:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
mkdir: Cannot create directory /user/mydata. Name node is in safe mode.
Now how to debug further?
681
The Hadoop has a variety of file systems that can be implemented concretely. The Java abstract class org.apache.hadoop.fs.FileSystem represents a file system in Hadoop. The Hadoop Local filesystem is used for a locally connected disk with client-side checksumming. The local filesystem uses RawLocalFileSystem with no checksums.
In Hadoop dfs there is no home directory by default. So let’s first create it. bin/hdfs dfs -mkdir /geeks => '/' means absolute path bin/hdfs dfs -mkdir geeks2 => Relative path -> the folder will be created relative to the home directory.
hdfs dfs -mkdir directoryName Create a new file in directory hdfs dfs -touchz directoryName/Newfilename Write into newly created file in HDFS nano filename Save it Cntr+XY Read the newly created file from HDFS nano fileName Or hdfs dfs -cat directoryName/fileName Share Improve this answer Follow edited Feb 15, 2018 at 11:29
To use the HDFS commands, first you need to start the Hadoop services using the following command: sbin/start-all.sh. To check the Hadoop services are up and running use the following command: jps. Commands: ls: This command is used to list all the files. Use lsr for recursive approach. It is useful when we want a hierarchy of a folder.
To leave safe mode, try below command since hadoop dfsadmin -safemode is deprecated in newer distribution of Hadoop:
hdfs dfsadmin -safemode leave
By default, user's home directory in hdfs exists with '/user/hduser' not as /home/hduser'.
If you tried to create directory directly like below then it will be created like '/user/hduser/sampleDir'.
hadoop fs -mkdir /path/to/be/created
On HDFS,
hdfs dfs -mkdir -p /this/is/a/new/directory
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
