Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

atomic file creation on Linux?

I need to create a file if it does not exist, in a way that another process trying to create this file would fail. I need the file be considered "created" even before the creating process finished writing the actual data to it.

I read about O_EXCL flag to open(), so it seems that the solution exists, I have a few questions however:

  1. do you have experience with this technique? How good is it? (I guess I can't have a DB-level atomicity, but but good enough is... well, enough)
  2. should I immediately close the file after open() so that it is considered created, and then reopen it for writing?
  3. are there any subtleties to be aware of?
like image 540
davka Avatar asked Feb 24 '23 21:02

davka


1 Answers

The open() man page says your method may fail on NFS.

From the section on O_EXCL:

When used with O_CREAT, if the file already exists it is an error and the open() will fail. In this context, a symbolic link exists, regardless of where it points to. O_EXCL is broken on NFS file systems; programs which rely on it for performing locking tasks will contain a race condition.

And it suggests a more general solution:

The solution for performing atomic file locking using a lockfile is to create a unique file on the same file system (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful.

See the "Using Files as Locks" section of this Web page for more details on the various issues and approaches.

like image 134
payne Avatar answered Feb 27 '23 10:02

payne