Menu
Is there an HDFS API that can copy an entire local directory to the HDFS? I found an API for copying files but is there one for directories?
620
You can use the cp command in Hadoop. This command is similar to the Linux cp command, and it is used for copying files from one directory to another directory within the HDFS file system.
Step 1: Make a directory in HDFS where you want to copy this file with the below command. Step 2: Use copyFromLocal command as shown below to copy it to HDFS /Hadoop_File directory. Step 3: Check whether the file is copied successfully or not by moving to its directory location with below command.
In order to copy a file from the local file system to HDFS, use Hadoop fs -put or hdfs dfs -put, on put command, specify the local-file-path where you wanted to copy from and then HDFS-file-path where you wanted to copy to. If the file already exists on HDFS, you will get an error message saying “File already exists”.
Use the Hadoop FS shell. Specifically:
$ hadoop fs -copyFromLocal /path/to/local hdfs:///path/to/hdfs If you want to do it programmatically, create two FileSystems (one Local and one HDFS) and use the FileUtil class
110
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
