Read file changes since the last check without running program constantly in C#

Question

I am trying to create a simple file monitor that checks for updates periodically in a log file and processes the updates. I tried using FileSystemWatcher but that requires my application to keep running forever. I am thinking more on the lines of read the changes, quit, wait for timer, read changes again.

I have created a service that runs periodically to read the file and get the whole data. Below is the simple code.

private void SchedularCallback(object e)
{
    string logFile = ReadFromFile("C:\\test.log");
    this.WriteToFile(logFile);
    this.ScheduleService();
}

WriteToFile function writes the data to a separate file (process data, actual process can involve other tasks like calling WCF services, checking internet access etc). ReadFromFile reads the log file every time the callback happens. Below is the code that reads the file.

private string ReadFromFile(string path)
{
    try
    {
        string logs = "";
        using (StreamReader reader = new StreamReader(path, true))
        {
            logs = reader.ReadToEnd();
            reader.Close();
        }

        return logs;
    }
    catch (Exception ex)
    {
        WriteToFile("Simple Service Error on: {0} " + ex.Message + ex.StackTrace);

        //Stop the Windows Service.
        using (System.ServiceProcess.ServiceController serviceController = new System.ServiceProcess.ServiceController("SimpleService"))
        {
           serviceController.Stop();
        }
        return "";
    }
}

As you notice, this code reads the whole file every time callback happens. Since log file can end up really big, reading and processing whole file every time is not possible. To improve this, I thought of using FileSystemWatcher, but that will keep my service running forever and just be a real performance drain. Instead if I can read just the changes in the file, it will be faster.

I also thought of storing the last offset of the streamwriter, but that will work only if data is appended. If someone deletes whole log or changes a line or two, last offset won't work.

In this case, what will be the best way. Log file obviously won't change constantly, so I don't need to keep my service running. I am unsure if a binarystream and then comparison with last binary stream will be a good idea. Any suggestion on a possible approach is appreciated. Basically something like what git does to identify changes since last commit, that's what I am looking for.

Thanks.

Answer 1

Have a look at the USN Journal for NTFS.

It basically logs all changes to files on an NTFS disk.

Here are some links which might prove useful:

1. Creating, Modifying, and Deleting a Change Journal
2. Fsutil usn
3. File Path from USN Journal