Adding content to middle of file..without reading it till the end

Question

I have read various questions/answers here on unix.stackexchange on how to add or remove lines to/from a file without needing to create a temporary file.

https://unix.stackexchange.com/questions/11067/is-there-a-way-to-modify-a-file-in-place?lq=1

It appears all these answers need one to atleast read till end of the file, which can be time consuming if input is a large file. Is there a way around this? I would expect a file system to be implemented like a linked list...so there should be a way to reach the required "lines" and then just add stuff (node in linked lists). How do I go about doing this?

Am I correct in thinking so? Or Am I missing anything?

Ps: I need this to be done in 'C' and cannot use any shell commands.

Answer 1

The short answer is that yes, it is possible to modify the contents of a file in place, but no, it is not possible to remove or add content in the middle of the file.

UNIX filesystems are implemented using an inode pointer structure which points to whole blocks of data. Each line of a text file does not "know" about its relationship to the previous or next line, they are simply adjacent to each other within the block. To add content between those two lines would require all of the following content to be shifted further "down" within the block, pushing some data into the next block, which in turn would have to shift into the following block, etc.

In C you can fopen a file for update and read its contents, and overwrite some of the contents, but I do not believe there is (even theoretically) any way to insert new data in the middle, or delete data (except to overwrite it with nulls.

Answer 2

As of Linux 4.1, fallocate(2) supports the FALLOC_FL_INSERT_RANGE flag, which allows one to insert a hole of a given length in the middle of a file without rewriting the following data. However, it is rather limited: the hole must be inserted at a filesystem block boundary, and the size of the inserted hole must be a multiple of the filesystem block size. Additionally, in 4.1, this feature was only supported by the XFS filesystem, with Ext4 support added in 4.2.

For all other cases, it is still necessary to rewrite the rest of the file, as indicated in other answers.

Answer 3

You can modify a file in place, for example using dd.

$ echo Hello world, how are you today? > helloworld.txt
$ cat helloworld.txt
Hello world, how are you today?
$ echo -n Earth | dd of=helloworld.txt conv=notrunc bs=1 seek=6
$ cat helloworld.txt
Hello Earth, how are you today?

The problem is that if your change also changes the length, it will not quite work correctly:

$ echo -n Joe | dd of=helloworld.txt conv=notrunc bs=1 seek=6
Hello Joeth, how are you today?
$ echo -n Santa Claus | dd of=helloworld.txt conv=notrunc bs=1 seek=6
Hello Santa Clausare you today?

When you change the length, you have to re-write the file, if not completely then starting at the point of change you make.

In C this is the same as with dd. You open the file, you seek, and you write.