How to force file flushing

Question

Suppose that I have the following code:

#include <chrono>
#include <fstream>
#include <thread>

int main()
{
  std::ofstream f("test.log");

  int i = 0;
  while (true)
  {
    f << i++;
    f.flush();

    std::this_thread::sleep_for(std::chrono::milliseconds(100));
  }
}

(note that I have a flush call after each write operation)

I noticed that this application doesn't update "last modified time" and "size" attributes of the "test.log" file unless I do a right-click on this file or open it.

I guess that this is due to an internal bufferization (system doesn't want to make such time-consuming operations as an actual I/O to disk unless forced to do so). Am I right?

I need to write an application that should watch for changes in log files created by other applications (I can't change them). At first, I thought about FileSystemWatcher class in C# but I noticed that it has the same behavior (it doesn't fire a corresponding event unless file was closed in a source application or was forced to update by right-clicking that file in Windows Explorer). What can I do then? Call WinAPI functions like GetFileAttributes for every file that I want to look for as often as I can?

Answer 1

There are two separate things here. First, the last modified time on the file MFT record (inode equivalent) is updated every time you write to it.

However the information returned by FindFirstFile and friends is not from the file, it is from information cached in the directory entry. This cache is updated whenever a file is closed which was opened through that directory entry. This is the information displayed by most applications, such as Windows Explorer, and the command prompt DIR command.

If you want to know when a file was updated you need to do the equivalent of a Unix stat operation which reads the MFT record (inode). This requires opening a handle to the file, calling GetFileInformationByHandle and closing the handle again.

The second thing is that there is a good reason not to do this. If a program is writing to a file, it may be partway through the writing process. Therefore the file may be in an invalid (corrupt) state. To ensure that the file is in a valid state you should wait until the file has been closed. This is how you know that the file is now ready to look at.

Once the writing program has finished writing to the file, the directory entry will be updated and FileSystemWatcher will show the file.

If you are absolutely sure you want to see notifications of files which are still in the process of being written, then you can look into the USN change journal as an option. I don't know if this is kept more up to date than the directory entries, you will have to investigate that.