Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to copy file from HDFS to the local file system

Tags:

copy

hadoop

hdfs

People also ask

How do I transfer files from HDFS to local?

Hadoop Get command is used to copy files from HDFS to the local file system, use Hadoop fs -get or hdfs dfs -get , on get command, specify the HDFS-file-path where you wanted to copy from and then local-file-path where you wanted a copy to the local file system. Copying files from HDFS file to local file system.

What command is used to copy a file from HDFS to local file system?

copyToLocal: Copy files from HDFS to local file system, similar to -get command.

Is it possible to move files from HDFS?

You can use the cp command in Hadoop. This command is similar to the Linux cp command, and it is used for copying files from one directory to another directory within the HDFS file system.


  1. bin/hadoop fs -get /hdfs/source/path /localfs/destination/path
  2. bin/hadoop fs -copyToLocal /hdfs/source/path /localfs/destination/path
  3. Point your web browser to HDFS WEBUI(namenode_machine:50070), browse to the file you intend to copy, scroll down the page and click on download the file.

In Hadoop 2.0,

hdfs dfs -copyToLocal <hdfs_input_file_path> <output_path>

where,

  • hdfs_input_file_path maybe obtained from http://<<name_node_ip>>:50070/explorer.html

  • output_path is the local path of the file, where the file is to be copied to.

  • you may also use get in place of copyToLocal.


In order to copy files from HDFS to the local file system the following command could be run:

hadoop dfs -copyToLocal <input> <output>

  • <input>: the HDFS directory path (e.g /mydata) that you want to copy
  • <output>: the destination directory path (e.g. ~/Documents)

Update: Hadoop is deprecated in Hadoop 3

use hdfs dfs -copyToLocal <input> <output>


you can accomplish in both these ways.

1.hadoop fs -get <HDFS file path> <Local system directory path>
2.hadoop fs -copyToLocal <HDFS file path> <Local system directory path>

Ex:

My files are located in /sourcedata/mydata.txt I want to copy file to Local file system in this path /user/ravi/mydata

hadoop fs -get /sourcedata/mydata.txt /user/ravi/mydata/