Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Atomically creating a file if it doesn't exist in Python

Tags:

python

atomic

I am looking for an atomic version of the following:

import os

def tryMakeFile(filename):
    try:
        with open(filename) as _:
            return False
    except FileNotFoundError:
        with open(filename, mode='a') as _:
            return True

(Please don't comment on stylistic issues here - I know this code is bad in many ways, but it suffices to illustrate my question.)

In other words, I'm looking for a way to check if a file exists, and create it if it doesn't, in Python, in such a way that I know which happened. But done in such a way that there isn't a race condition between multiple processes (in my given example code, two processes could both think they created the file, if the second process ran while the first was suspended between the first and second open calls).

Or, to put it another way, I am looking for a Python equivalent of Java's Files.createFile call.

Edit: note that when I say "Python" I mean "portable Python". Saying "use this library* (*this library is only available on Windows, or not on Windows, or only on the second Tuesday after a blue moon)" isn't what I'm looking for. I'm looking for something that is explicitly atomic, part of the standard library and/or builtins, and it's available on common platforms.

like image 776
TLW Avatar asked Oct 19 '15 20:10

TLW


2 Answers

You can use os.open with os.O_CREAT | os.O_EXCL flags which will fail if the file exists, they are according to the docs available on Unix and Windows but I am not sure if atomic file creation exists on windows or not:

os.open("filename", os.O_CREAT | os.O_EXCL)

From the linux open man page:

O_EXCL If O_CREAT and O_EXCL are set, open() shall fail if the file exists. The check for the existence of the file and the creation of the file if it does not exist shall be atomic with respect to other threads executing open() naming the same filename in the same directory with O_EXCL and O_CREAT set. If O_EXCL and O_CREAT are set, and path names a symbolic link, open() shall fail and set errno to [EEXIST], regardless of the contents of the symbolic link. If O_EXCL is set and O_CREAT is not set, the result is undefined.

Not sure what you want to do if the file exists but you just need to catch a FileExistsError when the file does already exist:

import os

def try_make_file(filename):
    try:
        os.open(filename,  os.O_CREAT | os.O_EXCL)
        return True
    except FileExistsError:
        return False
like image 61
Padraic Cunningham Avatar answered Sep 29 '22 11:09

Padraic Cunningham


If you have Python 3.3 or better, you could use the 'x' mode with open():

'x' open for exclusive creation, failing if the file already exists

def tryMakeFile(filename):
    try:
        with open(filename, "x") as _:
            return False
    except FileExistsError:
        return True
like image 27
Eugene Yarmash Avatar answered Sep 29 '22 10:09

Eugene Yarmash