How do I properly handle pollEvents() overflow kind in a watchservice?

Question

I am using a watch service to monitor a directory and fire logic on newly created files. One of the challenges that I've recently experienced is an overflow being triggered when a high volume of files needs to be processed and gets copied into the watch directory too quick to handle.

The API says this about overflow:

File systems may report events faster than they can be retrieved or processed and an implementation may impose an unspecified limit on the number of events that it may accumulate. Where an implementation knowingly discards events then it arranges for the key's pollEvents method to return an element with an event type of OVERFLOW.

My question is, how can I properly handle the overflow, without losing any of the events that need to be processed?

My watchservice code looks like:

            Path myDir = Paths.get(srcDir);
            try(WatchService watcher = myDir.getFileSystem().newWatchService()){
                myDir.register(watcher, ENTRY_CREATE,ENTRY_MODIFY);
                int x = 0;
                for(;;){
                    x++;
                    WatchKey watchKey = watcher.take();
                    LOGGER.debug("Event # {}",x);
                    List<WatchEvent<?>> events = watchKey.pollEvents();
                    LOGGER.info("Events Size {}",events.size());
                    for (WatchEvent event : events) {
                        if(event.kind() == OVERFLOW){
                            LOGGER.error("The Maximum watchService events has been reached!");
                            System.exit(1); //I exit so I know there is a problem - but how should I handle this? 
                        }
                        if (event.kind() == ENTRY_CREATE) {
                            LOGGER.info("File created: " + event.context().toString());
                            LOGGER.info("Beginning Processing:" +event.context().toString());
                            ...business logic here...
                        }
                    }
                    watchKey.reset();
                }
          ...

Answer 1

I've never seen the overflow event in practice. Its meant to inform you that you will need to reprocess whatever directory you're watching. You don't need to exit the program, just crawl the directory using a single threaded File.list() call. I've cut-n-pasted how I handle it below. This code...

1) Logs an issue

2) Sets a flag to trigger a directory reprocess that crawls all the files in a directory

3) Skips the rest of the processing of this WatchEvent

// log overflow events and trigger reprocess later.
if (kind == OVERFLOW) 
{
    logger.warn("File listener recieved an overflow event.  You should probably check into this");
    overflowTriggeredFlag = true;
    continue;
}