Copy binary data from URL to file in Java without intermediate copy

Question

I'm updating some old code to grab some binary data from a URL instead of from a database (the data is about to be moved out of the database and will be accessible by HTTP instead). The database API seemed to provide the data as a raw byte array directly, and the code in question wrote this array to a file using a BufferedOutputStream.

I'm not at all familiar with Java, but a bit of googling led me to this code:

URL u = new URL("my-url-string");
URLConnection uc = u.openConnection();
uc.connect();
InputStream in = uc.getInputStream();
ByteArrayOutputStream out = new ByteArrayOutputStream();
final int BUF_SIZE = 1 << 8;
byte[] buffer = new byte[BUF_SIZE];
int bytesRead = -1;
while((bytesRead = in.read(buffer)) > -1) {
    out.write(buffer, 0, bytesRead);
}
in.close();
fileBytes = out.toByteArray();

That seems to work most of the time, but I have a problem when the data being copied is large - I'm getting an OutOfMemoryError for data items that worked fine with the old code.

I'm guessing that's because this version of the code has multiple copies of the data in memory at the same time, whereas the original code didn't.

Is there a simple way to grab binary data from a URL and save it in a file without incurring the cost of multiple copies in memory?

Answer 1

Instead of writing the data to a byte array and then dumping it to a file, you can directly write it to a file by replacing the following:

ByteArrayOutputStream out = new ByteArrayOutputStream();

With:

FileOutputStream out = new FileOutputStream("filename");

If you do so, there is no need for the call out.toByteArray() at the end. Just make sure you close the FileOutputStream object when done, like this:

out.close();

See the documentation of FileOutputStream for more details.