C# - Waiting for a copy operation to complete

Question

I have a program that runs as a Windows Service which is processing files in a specific folder.

Since it's a service, it constantly monitors a folder for new files that have been added. Part of the program's job is to perform comparisons of files in the target folder and flag non-matching files.

What I would like to do is to detect a running copy operation and when it is completed, so that a file is not getting prematurely flagged if it's matching file has not been copied over to the target folder yet.

What I was thinking of doing was using the FileSystemWatcher to watch the target folder and see if a copy operation is occurring. If there is, I put my program's main thread to sleep until the copy operation has completed, then proceed to perform the operation on the folder like normal.

I just wanted to get some insight on this approach and see if it is valid. If anyone else has any other unique approaches to this problem, it would be greatly appreciated.

UPDATE:

I apologize for the confusion, when I say target directory, I mean the source folder containing all the files I want to process. A part of the function of my program is to copy the directory structure of the source directory to a destination directory and copy all valid files to that destination directory, preserving the directory structure of the original source directory, i.e. a user may copy folders containing files to the source directory. I want to prevent errors by ensuring that if a new set of folders containing more subfolders and files is copied to the source directory for processing, my program will not start operating on the target directory until the copy process has completed.

Answer 1

Yup, use a FileSystemWatcher but instead of watching for the created event, watch for the changed event. After every trigger, try to open the file. Something like this:

var watcher = new FileSystemWatcher(path, filter);
watcher.Changed += (sender, e) => {
    FileStream file = null;
    try {
        Thread.Sleep(100); // hack for timing issues
        file = File.Open(
            e.FullPath,
            FileMode.Open,
            FileAccess.Read,
            FileShare.Read
        );
    }
    catch(IOException) {
        // we couldn't open the file
        // this is probably because the copy operation is not done
        // just swallow the exception
        return;
    }

    // now we have a handle to the file
};

This is about the best that you can do, unfortunately. There is no clean way to know that the file is ready for you to use.

Answer 2

What you are looking for is a typical producer/consumer scenario. What you need to do is outlined in 'Producer/consumer queue' section on this page. This will allow you to use multi threading (maybe span a backgroundworker) to copy files so you don't block the main service thread from listening to system events & you can perform more meaningful tasks there - like checking for new files & updating the queue. So on main thread do check for new files on background threads perform the actual coping task. From personal experience (have implemented this tasks) there is not too much performance gain from this approach unless you are running on multiple CPU machine but the process is very clean & smooth + the code is logically separated nicely.

In short, what you have to do is have an object like the following:

public class File
{
    public string FullPath {get; internal set;}
    public bool CopyInProgress {get; set;} // property to make sure 
    // .. other properties if desired
}

Then following the tutorial posted above issue a lock on the File object & the queue to update it & copy it. Using this approach you can use this type approaches instead of constantly monitoring for file copy completion. The important point to realize here is that your service has only one instance of File object per actual physical file - just make sure you (1)lock your queue when adding & removing & (2) lock the actual File object when initializing an update.

EDIT: Above where I say "there is not too much performance gain from this approach unless" I refere to if you do this approach in a single thread compare to @Jason's suggesting this approach must be noticeably faster due to @Jason's solution performing very expensive IO operations which will fail on most cases. This I haven't tested but I'm quite sure as my approach does not require IO operations open(once only), stream(once only) & close file(once only). @Jason approach suggests multiple open,open,open,open operations which will all fail except the last one.

Answer 3

One approach is to attempt to open the file and see if you get an error. The file will be locked if it is being copied. This will open the file in shared mode so it will conflict with an already open write lock on the file:

using(System.IO.File.Open("file", FileMode.Open,FileAccess.Read, FileShare.Read)) {}

Another is to check the file size. It would change over time if the file is being copied to.

It is also possible to get a list of all applications that has opened a certain file, but I don't know the API for this.

Answer 4

I know this is an old question, but here's an answer I spun up after searching for an answer to just this problem. This had to be tweaked a lot to remove some of the proprietary-ness from what I was working on, so this may not compile directly, but it'll give you an idea. This is working great for me:

void BlockingFileCopySync(FileInfo original, FileInfo copyPath)
{
    bool ready = false;

    FileSystemWatcher watcher = new FileSystemWatcher();
    watcher.NotifyFilter = NotifyFilters.LastWrite;
    watcher.Path = copyPath.Directory.FullName;
    watcher.Filter = "*" + copyPath.Extension;
    watcher.EnableRaisingEvents = true;

    bool fileReady = false;
    bool firsttime = true;
    DateTime previousLastWriteTime = new DateTime();

    // modify this as you think you need to...
    int waitTimeMs = 100;

    watcher.Changed += (sender, e) =>
    {
        // Get the time the file was modified
        // Check it again in 100 ms
        // When it has gone a while without modification, it's done.
        while (!fileReady)
        {
            // We need to initialize for the "first time", 
            // ie. when the file was just created.
            // (Really, this could probably be initialized off the
            // time of the copy now that I'm thinking of it.)
            if (firsttime)
            {
                previousLastWriteTime = System.IO.File.GetLastWriteTime(copyPath.FullName);
                firsttime = false;
                System.Threading.Thread.Sleep(waitTimeMs);
                continue;
            }

            DateTime currentLastWriteTime = System.IO.File.GetLastWriteTime(copyPath.FullName);

            bool fileModified = (currentLastWriteTime != previousLastWriteTime);

            if (fileModified)
            {
                previousLastWriteTime = currentLastWriteTime;
                System.Threading.Thread.Sleep(waitTimeMs);
                continue;
            }
            else
            {
                fileReady = true;
                break;
            }
        }
    };

    System.IO.File.Copy(original.FullName, copyPath.FullName, true);

    // This guy here chills out until the filesystemwatcher 
    // tells him the file isn't being writen to anymore.
    while (!fileReady)
    {
        System.Threading.Thread.Sleep(waitTimeMs);
    }
}