Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Atomic file save on Linux without losing metadata

I'm working on a Perl-based file synchronization tool. It downloads files into a temporary directory (which is guaranteed to be on the same filesystem as the real file) and then moves the temporary files into place over the old ones, preserving metadata like permissions, ownership, and ACLs. I'm wondering how to achieve that last step on Linux.

On Mac OS X, at least in C, I would use the exchangedata function. This takes two filenames as arguments and swaps their contents, leaving all metadata (besides mtime) intact. It guarantees that the operation is atomic—all readers will see either the old file or the new one, never something in between. Unfortunately, I don't think it's available on Linux.

I know that rename moves atomically, but it doesn't preserve metadata. On the other hand, I could open the file and overwrite the data with the contents of the new one, which would preserve all metadata but would not be an atomic operation. Any suggestions on tackling this problem?

like image 787
Becca Royal-Gordon Avatar asked Nov 07 '09 18:11

Becca Royal-Gordon


People also ask

Is Fsync Atomic?

Sadly, fsync() is not atomic itself.

Is Linux move Atomic?

mv is most definitely not atomic when the move that it performs is from one filesystem to another, or when a remote filesystem cannot implement the mv operation locally. In these instances mv could be said to be implemented by the equivalent of a cp followed by rm .

What is atomic file update?

An atomic operation is one that changes a system from one state to another without visibly passing through any intermediate states. Atomicity is desirable when altering the content of a file because: The process performing the alteration may fail or be stopped, leaving the file in an incomplete or inconsistent state.

What are atomic files?

Atomic file guarantees file integrity by ensuring that a file has been completely written and sync'd to disk before renaming it to the original file.


2 Answers

The only approach I see here is to read the metadata from the file you are replacing, apply that to the temporary file, and then rename the temporary file over the old file. (rename preserves the source file attributes, obviously.)

like image 170
jrockway Avatar answered Sep 28 '22 06:09

jrockway


Filesystem-specific, but...

The XFS_IOC_SWAPEXT ioctl swaps the extents of two file descriptors on XFS.

#include <xfs/xfs.h>
#include <xfs/xfs_dfrag.h>

xfs_swapext_t sx = {
    ...,
    .sx_fdtarget = fd1,
    .sx_fdtmp    = fd2,
    ...
};
xfs_swapext(fd1, &sx);

See the sources to xfs_fsr for example usage.

like image 43
ephemient Avatar answered Sep 28 '22 05:09

ephemient