Premature IN_CLOSE_WRITE notification with pyinotify

Question

I am experiencing the following situation:

• pyinotify monitors a file for IN_CLOSE_WRITE events
• I change something in the file and save it
• the event is triggered
• I read the file and see that it has no changes

Having tinkered with this a little bit, I noticed that it works fine when I am debugging. I set a breakpoint on the line that reads the file, thus adding a little delay. After that - the file is read and the changes are there.

So, it seems that adding a time.sleep(1), or delaying execution in some other way does the trick. Otherwise I receive a premature IN_CLOSE_WRITE event.

I wondered whether the event is triggered after the changes are committed and the file is closed, or before that. There seem to be no other relevant events after IN_CLOSE_WRITE. At the same time, the documentation is a little bit tricky:

use IN_CLOSE_WRITE because if emitted the all changes on the appropriate file are safely written inside the file

I filed a bug report about the wording in the FAQ, but in the meantime I would like to get some additional input on this matter. Is this supposed to happen? What are the "morally correct" ways of solving it?

All of this is happening on a Linux Mint 15 x64.

Answer 1

It turns out that this behaviour is not an anomaly:

As I said previously I think that the mission of inotify (and therefore as reported by Pyinotify) is to signal you when a file is closed (more precisely when it is closed by its file descriptor), but obviously the kernel uses buffers and so the file data may not be written on disk immediately. For more details see the man (2) of the close() function:

A successful close does not guarantee that the data has been successfully saved to disk, as the kernel defers writes. It is not common for a file system to flush the buffers when the stream is closed. If you need to be sure that the data is physically stored use fsync(2). (It will depend on the disk hardware at this point.)

Bottom line you can not rely on IN_CLOSE_WRITE to be certain that your data has finished to be been written on disk.

In other words, it is not a premature notification, it comes at the right time; but the underlying mechanisms of the OS can keep doing something to that file.

Answer 2

The documentation says "was closed":

$ man 7 inotify|grep IN_CLOSE_WRITE
           IN_CLOSE_WRITE    File opened for writing was closed (*).