Is there an os-independent way to atomically overwrite a file?

Question

If the file already exists, I want to overwrite it. If it doesn't exist, I want to create it and write to it. I'd prefer to not have to use a 3rd party library like lockfile (which seems to handle all types of locking.)

My initial idea was to:

1. Write to a temporary file with a randomly generated large id to avoid conflict.
2. Rename the temp filename -> new path name.

Answer 1

os.Rename calls syscall.Rename which for Linux/UNIXs uses the rename syscall (which is atomic*). On Windows syscall.Rename calls MoveFileW which assuming the source and destination are on the same device (which can be arranged) and the filesystem is NTFS (which is often the case) is atomic*.

I would take care to make sure the source and destination are on the same device so the Linux rename does not fail, and the Windows rename is actually atomic. As Dave C mentions above creating your temporary file (usually using ioutil.TempFile) in the same directory as existing file is the way to go; this is how I do my atomic renames.

This works for me in my use case which is:

1. One Go process gets updates and renames files to swap updates in.
2. Another Go process is watching for file updates with fsnotify and re-mmaps the file when it is updated.

In the above use case simply using os.Rename has worked perfectly well for me.

Some further reading:

1. Is rename() atomic? "Yes and no. rename() is atomic assuming the OS does not crash...."
2. Is an atomic file rename (with overwrite) possible on Windows?

*Note: I do want to point out that when people talk about atomic filesystem file operations, from an application perspective, they usually mean the operation happens or does not happen (which journaling can help with) from the users perspective. If you are using atomic in the sense of an atomic memory operation, very few filesystem operations (outside of direct I/O [O_DIRECT] one block writes and reads with disk buffering disabled) can be considered truly atomic.