Monitoring a directory for new file creation without FileSystemWatcher

Question

I have to create a Windows service which monitors a specified folder for new files and processes it and moves it to other location.

I started with using FileSystemWatcher. My boss doesn't like FileSystemWatcher and wants me to use polling by using a Timer or any other mechanism other than FileSystemWatcher.

How can you monitor directorying without using FileSystemWatcher using .NET framework?

Answer 1

Actually, the FileWatcher component is not 100% "stable" from my experience over the years. Push enough files into a folder and you will lose some events. This is especially true if you monitor a file share, even if you increase the buffer size.

So, for all practical reasons, use FileWatcher together with a Timer which scans a folder for changes, for the most optimal solution.

Examples of creating Timer code should be in abundance if you google it. If you keep track of the last DateTime when the timer ran, then check the modified date of each file, and compare it to the date. Fairly simple logic.

The timer interval depends of how urgent the changes are for your system. But check every minute should be fine for many scenarios.

Answer 2

Using @Petoj's answer I've included a full windows service that polls every five minutes for new files. Its contrained so only one thread polls, accounts for processing time and supports pause and timely stopping. It also supports easy attaching of a debbugger on system.start

 public partial class Service : ServiceBase{


    List<string> fileList = new List<string>();

    System.Timers.Timer timer;


    public Service()
    {
        timer = new System.Timers.Timer();
        //When autoreset is True there are reentrancy problems.
        timer.AutoReset = false;

        timer.Elapsed += new System.Timers.ElapsedEventHandler(DoStuff);
    }


    private void DoStuff(object sender, System.Timers.ElapsedEventArgs e)
    {
       LastChecked = DateTime.Now;

       string[] files = System.IO.Directory.GetFiles("c:\\", "*", System.IO.SearchOption.AllDirectories);

       foreach (string file in files)
       {
           if (!fileList.Contains(file))
           {
               fileList.Add(file);

               do_some_processing();
           }
       }


       TimeSpan ts = DateTime.Now.Subtract(LastChecked);
       TimeSpan MaxWaitTime = TimeSpan.FromMinutes(5);

       if (MaxWaitTime.Subtract(ts).CompareTo(TimeSpan.Zero) > -1)
           timer.Interval = MaxWaitTime.Subtract(ts).TotalMilliseconds;
       else
           timer.Interval = 1;

       timer.Start();
    }

    protected override void OnPause()
    {
        base.OnPause();
        this.timer.Stop();
    }

    protected override void OnContinue()
    {
        base.OnContinue();
        this.timer.Interval = 1;
        this.timer.Start();
    }

    protected override void OnStop()
    {
        base.OnStop();
        this.timer.Stop();
    }

    protected override void OnStart(string[] args)
    {
       foreach (string arg in args)
       {
           if (arg == "DEBUG_SERVICE")
                   DebugMode();

       }

        #if DEBUG
            DebugMode();
        #endif

        timer.Interval = 1;
        timer.Start();
   }

   private static void DebugMode()
   {
       Debugger.Break();
   }

 }

Answer 3

At program startup, use Directory.GetFiles(path) to get the list of files.

Then create a timer, and in its elapsed event call hasNewFiles:

    static List<string> hasNewFiles(string path, List<string> lastKnownFiles)
    {
        List<string> files = Directory.GetFiles(path).ToList();
        List<string> newFiles = new List<string>();

        foreach (string s in files)
        {
            if (!lastKnownFiles.Contains(s))
                newFiles.Add(s);
        }

        return new List<string>();
    }

In the calling code, you'll have new files if:

    List<string> newFiles = hasNewFiles(path, lastKnownFiles);
    if (newFiles.Count > 0)
    {
        processFiles(newFiles);
        lastKnownFiles = newFiles;
    }

edit: if you want a more linqy solution:

    static IEnumerable<string> hasNewFiles(string path, List<string> lastKnownFiles)
    {
        return from f in Directory.GetFiles(path) 
               where !lastKnownFiles.Contains(f) 
               select f;
    }

    List<string> newFiles = hasNewFiles(path, lastKnownFiles); 
    if (newFiles.Count() > 0) 
    { 
        processFiles(newFiles); 
        lastKnownFiles = newFiles; 
    } 

Answer 4

You could use Directory.GetFiles():

using System.IO;

var fileList = new List<string>();

foreach (var file in Directory.GetFiles(@"c:\", "*", SearchOption.AllDirectories))
{
    if (!fileList.Contains(file))
    {
        fileList.Add(file);
        //do something
    }
}

Note this only checks for new files not changed files, if you need that use FileInfo