Using Directory.Delete() and Directory.CreateDirectory() to overwrite a folder

Question

In my WebApi action method, I want to create/over-write a folder using this code:

string myDir = "...";
if(Directory.Exists(myDir)) 
{
    Directory.Delete(myDir, true);
}
Directory.CreateDirectory(myDir);

// 1 - Check the dir 
Debug.WriteLine("Double check if the Dir is created: " + Directory.Exists(myDir));

// Some other stuff here...

// 2 - Check the dir again
Debug.WriteLine("Check again if the Dir still exists: " + Directory.Exists(myDir));

Issue

Strangely, sometimes right after creating the directory, the directory does not exist!

Sometimes when checking the dir for the first time (where the number 1 is); Directory.Exist() returns true, other times false. Same happens when checking the dir for the second time (where the number 2 is).

Notes

• None of this part of code throw any exception.
• Only can reproduce this when publishing the website on server. (Windows server 2008)
• Happens when accessing the same folder.

Questions

• Is this a concurrency issue race condition?
• Doesn't WebApi or the Operating System handle the concurrency?
• Is this the correct way to overwrite a folder?
• Should I lock files manually when we have many API requests to the same file?

Or in General:

• What's the reason for this strange behavior?

UPDATE:

• Using DirectoryInfo and Refresh() instead of Directory does not solve the problem.

• Only happens when the recursive option of Delete is true. (and the directory is not empty).

Answer 1

Many filesystem operations are not synchonous on some filesystems (in case of windows - NTFS). Take for example RemoveDirectory call (which is called by Directory.DeleteDirectory at some point):

The RemoveDirectory function marks a directory for deletion on close. Therefore, the directory is not removed until the last handle to the directory is closed.

As you see, it will not really delete directory until all handles to it are closed, but Directory.DeleteDirectory will complete fine. In your case that is also most likely such concurrency problem - directory is not really created while you executing Directory.Exists.

So, just periodically check what you need and don't consider filesystem calls in .NET to be synchronous. You can also use FileSystemWatcher in some cases to avoid polling.

EDIT: I was thinking how to reproduce it, and here is the code:

internal class Program {
    private static void Main(string[] args) {
        const string path = "G:\\test_dir";
        while (true) {         
            if (Directory.Exists(path))
                Directory.Delete(path);       
            Directory.CreateDirectory(path);   
            if (!Directory.Exists(path))
                throw new Exception("Confirmed");                 
        }            
    }        
}

You see that if all filesystem calls were synchronous (in .NET), this code should run without problem. Now, before running that code, create empty directory at specified path (preferrably don't use SSD for that) and open it with windows explorer. Now run the code. For me it either throws Confirmed (which exactly reproduces your issue) or throws on Directory.Delete saying that directory does not exist (almost the same case). It does it 100% of the time for me.

Here is another code which when running on my machine confirms that it's certainly possible for File.Exists to return true directly after File.Delete call:

internal class Program {
    private static void Main(string[] args) {
        while (true) {
            const string path = @"G:\test_dir\test.txt";
            if (File.Exists(path))
                File.Delete(path);
            if (File.Exists(path))
                throw new Exception("Confirmed");
            File.Create(path).Dispose();
        }
    }        
 }

To do this, I opened G:\test_dir folder and during execution of this code tried to open constantly appearing and disappearing test.txt file. After couple of tries, Confirmed exception was thrown (while I didn't create or delete that file, and after exception is thrown, it's not present on filesystem already). So race conditions are possible in multiple cases and my answer is correct one.

Answer 2

I wrote myself a little C# method for synchronous folder deletion using Directory.Delete(). Feel free to copy:

private bool DeleteDirectorySync(string directory, int timeoutInMilliseconds = 5000)
{
    if (!Directory.Exists(directory))
    {
        return true;
    }

    var watcher = new FileSystemWatcher
    {
        Path = Path.Combine(directory, ".."),
        NotifyFilter = NotifyFilters.DirectoryName,
        Filter = directory,
    };
    var task = Task.Run(() => watcher.WaitForChanged(WatcherChangeTypes.Deleted, timeoutInMilliseconds));

    // we must not start deleting before the watcher is running
    while (task.Status != TaskStatus.Running)
    {
        Thread.Sleep(100);
    }

    try
    {
        Directory.Delete(directory, true);
    }
    catch
    {
        return false;
    }

    return !task.Result.TimedOut;
}

Note that getting task.Result will block the thread until the task is finished, keeping the CPU load of this thread idle. So that is the point where it gets synchronous.