File existence check after a folder rename returns an incorrect value on UNC share

Question

On a Windows 7 (or server) box, we have a folder on a UNC share (cross machine UNC, not localhost). We rename that folder, and then check for the existence of a file at the new folder location. Even though it exists, it takes almost 5 seconds for File.Exists to return true on it.

Full repro can be found on https://github.com/davidebbo/NpmFolderRenameIssue. Here is the core code:

// This file doesn't exist yet
// Note that the presence of this existence check is what triggers the bug below!!
Console.WriteLine("Exists (should be false): " + File.Exists("test/test2/myfile"));

// Create a directory, with a file in it
Directory.CreateDirectory("test/subdir/test");
File.WriteAllText("test/subdir/test/myfile", "Hello");

// Rename the directory
Directory.Move("test/subdir/test", "test/test2");

var start = DateTime.UtcNow;

// List the files at the new location. Here, our file shows up fine
foreach (var path in Directory.GetFiles("test/test2"))
{
    Console.WriteLine(path);
}

for (; ; )
{
    // Now do a simple existence test. It should also be true, but when
    // running on a (cross machine) UNC share, it takes almost 5 seconds to become true!
    if (File.Exists("test/test2/myfile")) break;

    Console.WriteLine("After {0} milliseconds, test/test2/myfile doesn't show as existing",
        (DateTime.UtcNow - start).TotalMilliseconds);
    Thread.Sleep(100);
}

Console.WriteLine("After {0} milliseconds, test/test2/myfile correctly shows as existing!",
    (DateTime.UtcNow - start).TotalMilliseconds);

So it seems like the initial existence check causes the existence value to be cached, causing this bogus behavior.

Questions: what is the explanation for this? What's the best way to avoid it?

NOTE: this issue initially arose when using npm (Node Package Manager) on Windows. The code I have here is a C# port of the repro. See https://github.com/isaacs/npm/issues/2230 for the original Node/npm issue. The goal is to find a way to address it.

Answer 1

David, The redirector implements a negative "File Not Found" cache which prevents a client from flooding a server with file not found requests. The default cache time is 5 seconds but you can modify the FileNotFoundCacheLifetime registry value to control the cache or disable it by setting this value to 0.

Details: http://technet.microsoft.com/en-us/library/ff686200(v=WS.10).aspx

Answer 2

There are multiple levels of caching in network code. This could slow down the time the file existence finally shows up.

A solution would be not to use file shares but create a simple client/server architecture where the server returns the file existence from the local file system. That should really speed up item detection times.

My guess would be that if you tried to open the file even if File.Exists says it doesn't exist yet it should be opened correctly so you can use the server existence information. If that won't work you can simply add a download option to the server/client architecture.