Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How does one reclaim zeroed blocks of a sparse file?

Consider a sparse file with 1s written to a portion of the file.

I want to reclaim the actual space on disk for these 1s as I no longer need that portion of the sparse file. The portion of the file containing these 1s should become a "hole" as it was before the 1s were themselves written.

To do this, I cleared the region to 0s. This does not reclaim the blocks on disk.

How do I actually make the sparse file, well, sparse again?

This question is similar to this one but there is no accepted answer for that question.

Consider the following sequence of events run on a stock Linux server:

$ cat /tmp/test.c
#include <unistd.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>

int main(int argc, char **argv) {
    int fd;
    char c[1024];

    memset(c,argc==1,1024);

    fd = open("test",O_CREAT|O_WRONLY,0777);
    lseek(fd,10000,SEEK_SET);
    write(fd,c,1024);
    close(fd);

    return 0;
}

$ gcc -o /tmp/test /tmp/test.c

$ /tmp/test

$ hexdump -C ./test
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002710  01 01 01 01 01 01 01 01  01 01 01 01 01 01 01 01  |................|
*
00002b10

$ du -B1 test; du -B1 --apparent-size test
4096        test
11024       test

$ /tmp/test clear

$ hexdump -C ./test
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002b10

$ du -B1 test; du -B1 --apparent-size test
4096        test
11024       test

# NO CHANGE IN SIZE.... HMM....

EDIT -

Let me further qualify that I don't want to rewrite files, copy files, etc. If it is not possible to somehow free previously allocated blocks in situ, so be it, but I'd like to determine if such is actually possible or not. It seems like "no, it is not" at this point. I suppose I'm looking for sys_punchhole for Linux (discussions of which I just stumbled upon).

like image 523
z8000 Avatar asked Dec 30 '09 21:12

z8000


People also ask

Does NTFS support sparse files?

Support for sparse files is introduced in the NTFS file system as another way to make disk space usage more efficient. When sparse file functionality is enabled, the system does not allocate hard disk drive space to a file except in regions where it contains nonzero data.

What is the use of sparse file in big data?

The advantage of sparse files is that storage space is only allocated when actually needed: Storage capacity is conserved, and large files can occasionally be created even if insufficient free space for the original file is available on the storage media.

What is a sparse file record?

A file format that saves storage space by recording only actual data. Whereas regular files record empty fields as blank data or runs of nulls, a sparse file includes meta-data that describe where the runs of non-data are located. The reported file size is always the size of the entire file.


1 Answers

It appears as if linux have added a syscall called fallocate for "punching holes" in files. The implementations in individual filesystems seem to focus on the ability to use this for pre-allocating a larger continous number of blocks.

There is also the posix_fallocate call that only focus on the latter, and is not usable for hole punching.

like image 181
Christian Avatar answered Oct 18 '22 17:10

Christian