Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is lazy space allocation in Google File system

I was going through google file system (GFS) paper, It mentions that GFS uses Lazy space allocation to reduce internal fragmentation.
Can someone explain, how lazy space reduces internal fragmetation?

Source: http://research.google.com/archive/gfs-sosp2003.pdf

like image 906
user2328404 Avatar asked Aug 07 '13 17:08

user2328404


1 Answers

With lazy space allocation, the physical allocation of space is delayed as long as possible, until data at the size of the chunk size (in GFS's case, 64 MB according the 2003 paper) is accumulated. In other words, the decision process that precedes the allocation of a new chunk on disk, is heavily influenced by the size of the data that is to be written. This preference of waiting instead of allocating more chunks based on some other characteristic, minimizes the chance of internal fragmentation (i.e. unused portions of the 64 MB chunk).

In the Google paper, it also says: "Most chunks are full because most files contain many chunks, only the last of which may be partially filled." So, the same approach is applied to file creation.

It is analogous to this: http://duartes.org/gustavo/blog/post/how-the-kernel-manages-your-memory

like image 161
knopch1425 Avatar answered Oct 25 '22 18:10

knopch1425