which inotify event signals the completion of a large file operation?

Question

for large files or slow connections, copying files may take some time.

using pyinotify, i have been watching for the IN_CREATE event code. but this seems to occur at the start of a file transfer. i need to know when a file is completely copied - it aint much use if it's only half there.

when a file transfer is finished and completed, what inotify event is fired?

Answer 1

IN_CLOSE probably means the write is complete. This isn't for sure since some applications are bad actors and open and close files constantly while working with them, but if you know the app you're dealing with (file transfer, etc.) and understand its' behaviour, you're probably fine. (Note, this doesn't mean the transfer completed successfully, obviously, it just means that the process that opened the file handle closed it).

IN_CLOSE catches both IN_CLOSE_WRITE and IN_CLOSE_NOWRITE, so make your own decisions about whether you want to just catch one of those. (You probably want them both - WRITE/NOWRITE have to do with file permissions and not whether any writes were actually made).

There is more documentation (although annoyingly, not this piece of information) in Documentation/filesystems/inotify.txt.

Answer 2

For my case I wanted to execute a script after a file was fully uploaded. I was using WinSCP which writes large files with a .filepart extension till done.

I first started modifying my script to ignore files if they're themselves ending with .filepart or if there's another file existing in the same directory with the same name but .filepart extension, hence that means the upload is not fully completed yet.

But then I noticed at the end of the upload, when all the parts have been finished, I have a IN_MOVED_IN notification getting triggered which helped me run my script exactly when I wanted it.

If you want to know how your file uploader behaves, add this to the incrontab:

/your/directory/ IN_ALL_EVENTS echo "$$ $@ $# $% $&"

and then

tail -F /var/log/cron

and monitor all the events getting triggered to find out which one suits you best.

Good luck!