Write to file without zero fill in Linux

Question

Let's say that the intent is to create a file with a large hole at the beginning that we will write to later on, on an embedded device running Linux. We open the file, obtain a file descriptor and call lseek on it to seek to a certain, known position. Afterwards, when we want to write to that file at the seeked-to position, we call write on it.

However, on the first write the hole created by seeking gets zero-filled and if the hole is large enough, this operation can take some time. In my application, there is no need for this zero initialization, as the hole is of exact length and I will fill it with my data later.

Is there a way to avoid having the first write call after seek zero-fill the hole (even if it involves modifying the filesystem driver)? Alternatively, is there a way of writing to a file before the beginning of the file (appending to the front of the file)?

Answer 1

This is likely related to your filesystem. On ext2/3/4, reiser, btrfs, xfs, and the like, doing what you describe should not take a long time, because they support what are called "sparse files" (files which take up less space in the underlying storage than the size of the file, because runs of zeroes aren't physically stored).

You might try an experiment with dd to make sure this is the case:

$ dd if=/dev/zero of=whatever bs=1k seek=1073741824 count=1
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 9.1878e-05 s, 11.1 MB/s
$ ls -al whatever
-rw-r--r-- 1 xxxx xxxx 1099511628800 Jan 31 18:04 whatever
$ du -h whatever
16K whatever

On your filesystem, that probably fails. If so, and what you need is to create a sparse file, determine whether you can use a different filesystem.

Answer 2

However, on the first write the hole created by seeking gets zero-filled and if the hole is large enough, this operation can take some time.

No it can't. It will just write the data you provided to write(). The zeros in the unwritten portion aren't physically there: they are an artefact of the file system.