Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Create file in a thread-safe manner

I have an array of filenames and each process need to create and write only to a single file.

This is what I came to:

foreach ($filenames as $VMidFile) {
    if (file_exists($VMidFile)) { // A
        continue;
    }

    $fp = fopen($VMidFile, 'c'); // B

    if (!flock($fp, LOCK_EX | LOCK_NB)) { // C
        continue;
    }

    if (!filesize($VMidFile)) { // D
        // write to the file;

        flock($fp, LOCK_UN);
        fclose($fp);
        break;
    }

    flock($fp, LOCK_UN);
    fclose($fp); // E
}

But I don't like that I'm relying on the filesize.

Any proposals to do it in another (better) way?

UPD: added the labels to discuss easily

UPD 2: I'm using filesize because I don't see any other reliable way to check if the current thread created the file (thus it's empty yet)

UPD 3: the solution should be condition race free.

like image 572
zerkms Avatar asked Jan 24 '13 20:01

zerkms


2 Answers

A possible, slightly ugly solution would be to lock on a lock file and then testing if the file exists:

$lock = fopen("/tmp/".$filename."LOCK", "w"); // A

if (!flock($lock, LOCK_EX)) { // B
    continue;
}
if(!file_exists($filename)){ // C
    //File doesn't exist so we know that this thread will create it
    //Do stuff to $filename
    flock($lock, LOCK_UN); // D
    fclose($lock);
}else{
    //File exists. This thread didn't create it (at least in this iteration).
    flock($lock, LOCK_UN);
    fclose($lock);
}

This should allow exclusive access to the file and also allows deciding whether the call to fopen($VMidFile, 'c'); will create the file.

like image 171
Jim Avatar answered Oct 12 '22 14:10

Jim


Rather than creating a file and hoping that it's not interfered with:

  1. create a temporary file
  2. do all necessary file operations on it
  3. rename it to the new location if location doesn't exist.

Technically, since rename will overwrite the destination there is a chance that concurrent threads will still clash. That's very unlikely if you have:

if(!file_exists($lcoation) { rename(...

You could use md5_file to verify the file contents is correct after this block.

like image 28
Hamish Avatar answered Oct 12 '22 12:10

Hamish