How to safely write to a file?

Question

Imagine you have a library for working with some sort of XML file or configuration file. The library reads the whole file into memory and provides methods for editing the content. When you are done manipulating the content you can call a write to save the content back to file. The question is how to do this in a safe way.

Overwriting the existing file (starting to write to the original file) is obviously not safe. If the write method fails before it is done you end up with a half written file and you have lost data.

A better option would be to write to a temporary file somewhere, and when the write method has finished, you copy the temporary file to the original file.

Now, if the copy somehow fails, you still have correctly saved data in the temporary file. And if the copy succeeds, you can remove the temporary file.

On POSIX systems I guess you can use the rename system call which is an atomic operation. But how would you do this best on a Windows system? In particular, how do you handle this best using Python?

Also, is there another scheme for safely writing to files?

Answer 1

If you see Python's documentation, it clearly mentions that os.rename() is an atomic operation. So in your case, writing data to a temporary file and then renaming it to the original file would be quite safe.

Another way could work like this:

• let original file be abc.xml
• create abc.xml.tmp and write new data to it
• rename abc.xml to abc.xml.bak
• rename abc.xml.tmp to abc.xml
• after new abc.xml is properly put in place, remove abc.xml.bak

As you can see that you have the abc.xml.bak with you which you can use to restore if there are any issues related with the tmp file and of copying it back.