How to avoid a race condition with makedirs?

Question

I am attempting to convert the following pseudo-code to Python:

If <directory> does not exist:
    Create all subdirectories for <directory>
Create a file in <directory>

This sounds simple enough to accomplish with os.makedirs and os.path.isdir:

if not os.path.isdir('/some/path'):
    os.makedirs('/some/path')
open('/some/path/test.txt', 'w')

However, upon further inspection there is clearly a race condition present. Consider the following timeline:

1. the specified directory (/some/path) does not exist
2. the Python interpreter executes the first line, which evaluates to True
3. another process creates the directory (/some/path)
4. makedirs raises an OSError exception since the directory already exists

There are also problems if the directory does initially exist but is removed by another process before the final line is executed.

When it comes to Python, "it's easier to ask for forgiveness than permission." With that in mind, the fragment above could be better written:

try:
    os.makedirs('/some/path')
except OSError:
    pass
open('/some/path/test.txt', 'w')

This solves the two problems described above but creates a third: os.makedirs raises an OSError exception when one of the following conditions occurs:

• the directory already exists
• the directory could not be created

This means that there is no way to determine which of the two conditions caused the exception to be raised. In other words, actual failures will be silently ignored, which is not what I want.

How can I work around this problem?

Answer 1

I'll note that all of this is quite a bit more sane in python 3; FileExistsError and PermissionError are separate (subclass of OSError) exceptions that you can catch, and os.makedirs even has a exist_ok kwarg to suppress the former when you're ok with the directory already existing.

If you want to inspect the reason for the OSError, that info is in a tuple in e.args (or optionally e.errno if you just want to look at the error code):

try:
    os.makedirs('/etc/python')
except OSError as e:
    print e.args

(17, 'File exists')

try:
    os.makedirs('/etc/stuff')
except OSError as e:
    print e.args

(13, 'Permission denied')

try:
    os.makedirs('/etc/stuff')
except OSError as e:
    print e.errno

13

So you'll have to do a bit of introspection and handle the two error codes differently in your except block.