Groovy: Appending to large files

Question

How do I append to large files efficiently. I have a process that has to continually append to a file and as the file size grows the performance seems to slow down as well. Is there anyway to specify a large buffer size with the append

Answer 1

While Don's approach is valid, in general (it will throw an exception, though, because of a syntax error, and you need to flush() a BufferedOutputStream), I've been planning to elaborate further (which takes its time).

Groovy does not provide special objects for I/O operations. Thus, you would use Java's FileOutputStream (for writing bytes) or FileWriter (for writing strings). Both provide a constructor that takes a boolean append parameter.
For both, there exist decorators (BufferedOutputStream and BufferedWriter), which are buffered. "Buffering", in this scope, means, that contents are not necessarily written instantly to the underlying stream, and thus, there's a potential for I/O optimization.

Don already provided a sample for the BufferedOutputStream, and here's one for the BufferedWriter:

File file = new File("foo")
if (file.exists()) {
    assert file.delete()
    assert file.createNewFile()
}

boolean append = true
FileWriter fileWriter = new FileWriter(file, append)
BufferedWriter buffWriter = new BufferedWriter(fileWriter)

100.times { buffWriter.write "foo" }

buffWriter.flush()
buffWriter.close()

While Groovy does not provide its own I/O objects, the Groovy JDK (GDK) enhances several Java types by adding convenience methods. In the scope of I/O outputting, the OutputStream and the File types are relevant.

So, finally, you can work with those the "Groovy way":

new File("foo").newOutputStream().withWriter("UTF-8") { writer ->
    100.times { writer.write "foo" + it }
}

---

EDIT: As per your further inquiry:

None of the GDK methods allows for setting a buffer size.

The above "Groovy" code will overwrite the file if called repeatedly. - In contrast, the following piece of code will append the string to an existing file and, thus, can be called repeatedly:

new File("foo").withWriterAppend("UTF-8") { it.write("bar") }