Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

When I store files in HDFS, will they be replicated?

Tags:

hadoop

hdfs

I am new to Hadoop.

When I store Excel files using hadoop -fs put commoad, it is stored in HDFS.

Replication factor is 3.

My question is: Does it take 3 copies and store them into 3 nodes each?

like image 372
user3017115 Avatar asked Dec 11 '22 09:12

user3017115


2 Answers

Here is a comic for HDFS working.

https://docs.google.com/file/d/0B-zw6KHOtbT4MmRkZWJjYzEtYjI3Ni00NTFjLWE0OGItYTU5OGMxYjc0N2M1/edit?pli=1

like image 99
Anju Singh Avatar answered Jan 26 '23 05:01

Anju Singh


Does it take 3 copies and store them into 3 nodes each.

answer is: NO

enter image description here

Replication is done in pipelining that is it copies some part of file to datanode1 and then copies to datanode2 from datanode1 and to datanode3 from datanode1

http://hadoop.apache.org/docs/r1.2.1/hdfs_design.html#Replication+Pipelining

see here for Replication Pipelining

like image 42
Nagendra kumar Avatar answered Jan 26 '23 04:01

Nagendra kumar