Use java.nio to raise create/move/delete events on file changes

Question

For a media library where I'd like to update entries when they change in the file system I wanted to give the 'new' java.nio file watching features a try by using this example.
I was expecting to get useful events when a file is being created, moved (renamed) or deleted, but here is what's happening when watching folders on windows7 (haven't tried the other operating systems yet):

Format:
[ThreadName] DEBUG 2012-04-09 18:20:35.934 GroupNumber-COMMAND: Path
ThreadName: Each watch folder runs in its own thread having a distinct id
GroupNumber: Messages sent with the same GroupNumber are being sent at the same time (usually..)
COMMAND: The received command
Path: The received path

Rename:
[Watch0] DEBUG 2012-04-09 18:20:35.934 2-ENTRY_DELETE: C:\tmp\tmp\test.avi
[Watch0] DEBUG 2012-04-09 18:20:35.935 2-ENTRY_CREATE: C:\tmp\tmp\test1.avi
[Watch0] DEBUG 2012-04-09 18:20:35.936 3-ENTRY_MODIFY: C:\tmp\tmp
[Watch0] DEBUG 2012-04-09 18:20:35.937 4-ENTRY_MODIFY: C:\tmp\tmp\test1.avi

[Watch4] DEBUG 2012-04-09 18:43:47.965 18-ENTRY_DELETE: F:\tmp\test.avi
[Watch4] DEBUG 2012-04-09 18:43:47.966 18-ENTRY_CREATE: F:\tmp\test1.avi
[Watch4] DEBUG 2012-04-09 18:43:47.967 19-ENTRY_MODIFY: F:\tmp\test1.avi

Create:
[Watch0] DEBUG 2012-04-09 18:22:02.055 5-ENTRY_CREATE: C:\tmp\test.avi
[Watch0] DEBUG 2012-04-09 18:22:02.066 6-ENTRY_MODIFY: C:\tmp\test.avi
[Watch0] DEBUG 2012-04-09 18:22:03.460 7-ENTRY_MODIFY: C:\tmp\test.avi
//Note the 1.4'' delay between the last two messages. 
//This is the time required to actually copy the file

Move in same watch folder:
[Watch0] DEBUG 2012-04-09 18:18:42.395 0-ENTRY_DELETE: C:\tmp\test.avi
[Watch0] DEBUG 2012-04-09 18:18:42.396 0-ENTRY_MODIFY: C:\tmp\tmp
[Watch0] DEBUG 2012-04-09 18:18:42.396 1-ENTRY_CREATE: C:\tmp\tmp\test.avi
[Watch0] DEBUG 2012-04-09 18:18:42.396 1-ENTRY_MODIFY: C:\tmp\tmp\test.avi

Move to other watch folder on same drive:
[Watch1] DEBUG 2012-04-09 18:23:24.341 8-ENTRY_CREATE: C:\tmp2\test.avi
[Watch0] DEBUG 2012-04-09 18:23:24.341 8-ENTRY_DELETE: C:\tmp\test.avi
[Watch1] DEBUG 2012-04-09 18:23:24.342 10-ENTRY_MODIFY: C:\tmp2\test.avi
//The two 8 are lying. Both messages are being sent from different threads
//and the shared counter hasn't been incremented by any yet. The next entry has been 
//incremented by two!

Move to other watch folder on different drive:
[Watch4] DEBUG 2012-04-09 18:25:42.324 11-ENTRY_CREATE: F:\tmp\test.avi
[Watch4] DEBUG 2012-04-09 18:25:42.338 12-ENTRY_MODIFY: F:\tmp\test.avi
[Watch4] DEBUG 2012-04-09 18:25:42.703 13-ENTRY_MODIFY: F:\tmp\test.avi
[Watch3] DEBUG 2012-04-09 18:25:49.433 14-ENTRY_DELETE: C:\tmp2\test.avi
//Note that the last delete message is being sent from another thread then the first ones.
//This is because the source and destination WatchDirs aren't the same

Delete:
[Watch9] DEBUG 2012-04-05 21:22:02.921 ENTRY_DELETE: C:\tmp\test (2011).mkv

Instead of having a single event, there are a set of 'command + path' that have to be interpreted. E.g. a delete consists of a single command, whereas rename and 'move in same folder' start with a delete command as well but will be defined by their future commands. Additionally, multiple files could be e.g. moved in parallel, which will end up in a random list of commands belonging to different operations having to be sorted somehow.

The best I could come up with is this class, where events are being enqueued when being received and then checked a moment (1s) after having been received in another thread (to give some time if other events are being generated and belong to the same 'event group').

This works if renaming, moving, creating or deleting a single file, but nothing will work anymore if starting to copy multiple files in parallel or copy a batch of files.

Does any implementation of what I need already exist (seems like a common use case)? Or has someone a good idea how to approach this problem to cover all cases?

In the end, it will have to work for windows, linux and osx.

A more complex example which should be supported as well

[Watch0] DEBUG 2012-04-09 19:10:17.774 0-ENTRY_CREATE: C:\tmp\tmp\testlarge.avi
[Watch0] DEBUG 2012-04-09 19:10:17.825 0-ENTRY_MODIFY: C:\tmp\tmp\testlarge.avi
[Watch0] DEBUG 2012-04-09 19:10:17.826 1-ENTRY_MODIFY: C:\tmp\tmp
[Watch0] DEBUG 2012-04-09 19:12:09.516 2-ENTRY_DELETE: C:\tmp\tmp\testsmall.avi
[Watch0] DEBUG 2012-04-09 19:12:09.516 3-ENTRY_CREATE: C:\tmp\testsmall.avi
[Watch0] DEBUG 2012-04-09 19:12:09.517 3-ENTRY_MODIFY: C:\tmp\testsmall.avi
[Watch0] DEBUG 2012-04-09 19:12:09.521 4-ENTRY_MODIFY: C:\tmp\tmp
[Watch0] DEBUG 2012-04-09 19:14:13.025 5-ENTRY_MODIFY: C:\tmp\tmp\testlarge.avi

Here, a small file is being moved while a large file is being created.

Answer 1

Not very familiar with java nio file watcher, but I used jnotify library to get events from the file system, it was very good and was working with windows and linux.

If you want to use nio any way try to take a look here: https://blogs.oracle.com/thejavatutorials/entry/watching_a_directory_for_changes