Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

In Java, what is the best/safest pattern for monitoring a file being appended to?

Tags:

java

file

nio

Someone else's process is creating a CSV file by appending a line at a time to it, as events occur. I have no control over the file format or the other process, but I know it will only append.

In a Java program, I would like to monitor this file, and when a line is appended read the new line and react according to the contents. Ignore the CSV parsing issue for now. What is the best way to monitor the file for changes and read a line at a time?

Ideally this will use the standard library classes. The file may well be on a network drive, so I'd like something robust to failure. I'd rather not use polling if possible - I'd prefer some sort of blocking solution instead.

Edit -- given that a blocking solution is not possible with standard classes (thanks for that answer), what is the most robust polling solution? I'd rather not re-read the whole file each time as it could grow quite large.

like image 611
Nick Fortescue Avatar asked Jul 02 '09 09:07

Nick Fortescue


People also ask

How do you append to a file in Java?

We can append to file in java using following classes. If you are working on text data and the number of write operations is less, use FileWriter and use its constructor with append flag value as true . If the number of write operations is huge, you should use the BufferedWriter.

How to open file in append mode in Java?

In Java, we can append a string in an existing file using FileWriter which has an option to open a file in append mode. Java FileWriter class is used to write character-oriented data to a file. It is a character-oriented class that is used for file handling in Java.


2 Answers

Since Java 7 there has been the newWatchService() method on the FileSystem class.

However, there are some caveats:

  • It is only Java 7
  • It is an optional method
  • it only watches directories, so you have to do the file handling yourself, and worry about the file moving etc

Before Java 7 it is not possible with standard APIs.

I tried the following (polling on a 1 sec interval) and it works (just prints in processing):

  private static void monitorFile(File file) throws IOException {
    final int POLL_INTERVAL = 1000;
    FileReader reader = new FileReader(file);
    BufferedReader buffered = new BufferedReader(reader);
    try {
      while(true) {
        String line = buffered.readLine();
        if(line == null) {
          // end of file, start polling
          Thread.sleep(POLL_INTERVAL);
        } else {
          System.out.println(line);
        }
      }
    } catch(InterruptedException ex) {
     ex.printStackTrace();
    }
  }

As no-one else has suggested a solution which uses a current production Java I thought I'd add it. If there are flaws please add in comments.

like image 169
Nick Fortescue Avatar answered Oct 19 '22 10:10

Nick Fortescue


You can register to get notified by the file system if any change happens to the file using WatchService class. This requires Java7, here the link for the documentation http://docs.oracle.com/javase/tutorial/essential/io/notification.html

here the snippet code to do that:

public FileWatcher(Path dir) {
   this.watcher = FileSystems.getDefault().newWatchService();
   WatchKey key = dir.register(watcher, ENTRY_MODIFY);
}

void processEvents() {
    for (;;) {
        // wait for key to be signalled
        WatchKey key;
        try {
            key = watcher.take();
        } catch (InterruptedException x) {
            return;
        }

        for (WatchEvent<?> event : key.pollEvents()) {
            WatchEvent.Kind<?> kind = event.kind();

            if (kind == OVERFLOW) {
                continue;
            }
            // Context for directory entry event is the file name of entry
            WatchEvent<Path> ev = cast(event);
            Path name = ev.context();
            Path child = dir.resolve(name);
            // print out event
            System.out.format("%s: %s file \n", event.kind().name(), child);
        }
        // reset key and remove from set if directory no longer accessible
        boolean valid = key.reset();
    }
}
like image 43
Ramcis Avatar answered Oct 19 '22 09:10

Ramcis