How to rewrite a file from a shell script without any danger of truncating the file if out of disk space?

Question

How to rewrite a file from a shell script without any danger of truncating the file if out of disk space?

This handy perl one liner replaces all occurrences of "foo" with "bar" in a file called test.txt:

perl -pi -e 's/foo/bar/g' test.txt

This is very useful, but ...

If the file system where test.txt resides has run out of disk space, test.txt will be truncated to a zero-byte file.

Is there a simple, race-condition-free way to avoid this truncation occuring?

I would like the test.txt file to remain unchanged and the command to return an error if the file system is out of space.

Ideally the solution should be easily used from a shell script without requiring additional software to be installed (beyond "standard" UNIX tools like sed and perl).

Thanks!

Answer 1

In general, this can’t be done. Remember that the out-of-space condition can hit anywhere along the sequence of actions that give the appearance of in-place editing. Once the filesystem is full, perl may not be able to undo previous actions in order to restore the original state.

A safer way to use the -i switch is to use a nonempty backup suffix, e.g.,

perl -pi.bak -e 's/foo/bar/g' test.txt

This way, if something goes wrong along the way, you still have your original data.

If you want to roll your own, be sure to check the value returned from the close system call. As the Linux manual page notes,

Not checking the return value of close() is a common but nevertheless serious programming error. It is quite possible that errors on a previous write(2) operation are first reported at the final close(). Not checking the return value when closing the file may lead to silent loss of data. This can especially be observed with NFS and with disk quota.

As with everything else in life, leave yourself more margin for error. Disk is cheap. Dig out the pocket change from your couch cushions and go buy yourself half a terabyte or so.