Why do FindFirstFile / FindNextFile sometimes see a file on a network folder, but CopyFile does not?

Question

I have a function that is supposed to create a backup of a source folder by copying all files from the source to a destination folder. The function uses a while loop driven by FindFirstFile / FindNextFile, then invokes CopyFile for each file found by the Find... function.

Now when the source folder is an SMB network path (regardless of whether I use a mapped drive or a UNC path), it sometimes happens that FindNextFile "sees" a file, but CopyFile refuses to copy the file. The error code is 2, i.e. ERROR_FILE_NOT_FOUND.

I found this hard to believe, so I added a call to _access to the backup function that checks for file existence just before CopyFile is called. The result is the same as for CopyFile, i.e. _access reports that the file does not exist (return code -1 and errno is 2, i.e. ENOENT).

So my main question is: How is it possible that FindFirstFile / FindNextFile "sees" files on a network folder that neither CopyFile nor _access can see?

Additional information / diagnostics:

• The problematic file is a file that is created immediately before the backup function is run. Specifically it works like this: A process that runs on the client machine has a network connection to a process that runs on the server machine. The client process tells the server process to create the file. This is supposed to work synchronously: Only after the server process acknowledges that it has created the file does the client process go on to perform the backup.
• I added a retry mechanism to the backup function. With this, CopyFile and_access suddenly start to see the file in question after about 4 seconds of retrying. This indicates to me that there is indeed some sort of network-delay involved. It appears as if FindFirstFile / FindNextFile access the network path differently than CopyFile / _access.

Unfortunately my research to this effect did not turn up any useful information, so I can only speculate. If FindFirstFile / FindNextFile / CopyFile indeed do not work well together, do you know of a different set of find/copy API functions that work more reliable?

Answer 1

This is likely due to the fact that SMB 2.0 on windows maintains a cache which only refreshes every 10 seconds.

See this blog ^archive article for more information and programmatic work arounds.

File.Exists /_access / GetFileAttributes / FindFirstFile,FindNextFile / _stat behavior over SMB 2.0

What is really happening?

This is because of the local caching included on the client side with SMB 2.0. When the SMB 2.0 session has been created, the local cache will be available on the client side, which will be refreshed after every 10 seconds by default. Any further request to the file exists will be checked against this local cache rather than going to the server share. So, if a local cache is built on client, and on server share a new file is created, local cache has not been invalidated and not in sync with server share, any further request for checking the new file existence will fail.

This is root cause, but by design. If the local cache is updated and in sync with server share, request will be successful.

For resolution/work around, the linked article mentions:

How do I control the local cache lifetime?

You can create registry keys below to control the cache lifetime.
Under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters:

• FileInfoCacheLifetime
• FileNotFoundCacheLifetime
• DirectoryCacheLifetime

They are all REG_DWORD type in seconds.

Programmatic workaround?

Register for directory or file change notifications using Win32 API.
Use FindFirstChangeNotification Function (Windows) API to register for changes.