On which systems/filesystems is os.open() atomic?

Question

This article states, that

fd = os.open('foo.lock', os.O_CREAT|os.O_EXCL|os.O_RDWR)

"is atomic on most filesystems". Is that true (on Unix and Windows)? On which filesystems?

The docs state that mentioned flags are available on Unix and Windows, so it looks like a tempting, cross-platform method for file locking (the flags O_CREAT and O_EXCL ensure that the calling process creates the file).

Answer 1

For UN*X-compliant (certified POSIX / IEEE 1003.1 as per the OpenGroup) systems, the behaviour is guaranteed as the OpenGroups specs for open(2) mandate this. Quote:

O_EXCL
If O_CREAT and O_EXCL are set, open() shall fail if the file exists. The check for the existence of the file and the creation of the file if it does not exist shall be atomic with respect to other threads executing open() naming the same filename in the same directory with O_EXCL and O_CREAT set. If O_EXCL and O_CREAT are set, and path names a symbolic link, open() shall fail and set errno to [EEXIST], regardless of the contents of the symbolic link. If O_EXCL is set and O_CREAT is not set, the result is undefined.

The "common" UN*X and UN*X-like systems (Linux, MacOSX, *BSD, Solaris, AIX, HP/UX) surely behave like that.

Since the Windows API doesn't have open() as such, the library function there is necessarily reimplemented in terms of the native API but it's possible to maintain the semantics.

I don't know which widely-used systems wouldn't comply; QNX, while not POSIX-certified, has the same statement in its docs for open(). The *BSD manpages do not explicitly mention the "atomicity" but Free/Net/OpenBSD implement it. Even exotics like SymbianOS (which like Windows doesn't have a UN*X-ish open system call) can do the atomic open/create.

For more interesting results, try to find an operating system / C runtime library which has open() but doesn't implement the above semantics for it... and on which Python would run with threads (got you there, MSDOS ...).

Edit: My post particularly focuses on "which operating systems have this characteristic for open ?" - for which the answer is, "pretty much all of them". Wrt. to filesystems though, the picture is different because network filesystems - whether NFS, SMB/CIFS or others, do not always maintain O_EXCL as this could result in denial-of-service (if a client does an open(..., O_EXCL, ...) and then simply stops talking with the fileserver / is shut down, everyone else would be locked out).