Monitoring files - how to know when a file is complete

Question

We have several .NET applications that monitor a directory for new files, using FileSystemWatcher. The files are copied from another location, uploaded via FTP, etc. When they come in, the files are processed in one way or another. However, one problem that I have never seen a satisfactory answer for is: for large files, how does one know when the files being monitored are still being written to? Obviously, we need to wait until the files are complete and closed before we begin processing them. The event args in the FileSystemWatcher events do not seem to address this.

Answer 1

If you are in control on the program that is writing the files into the directory, you can have the program write the files to a temporary directory and then move them into the watched directory. The move should be an atomic operation, so the watcher shouldn't see the file until it is fully in the directory.

If you are not in control of what is writing to the watched directory, you can set a time in the watcher where the file is considered complete when it has remained the same size for the given time. If immediate processing isn't a concern, setting this timer to something relatively large is a fairly safe way to know that either the file is complete or it never will be.

Answer 2

The "Changed" event on the FileSystemWatcher should shouldn't fire until the file is closed. See my answer to a similar question. There is a possibility that the FTP download mechanism closes the file multiple times during download as new data comes in, but I would think that is a little unlikely.