Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to write a potentially huge InputStream to File?

I have an API call that returns a byte array. I currently stream the result into a byte array then make sure the checksums match and then write the ByteArrayOutputStream to File. The code is something like this and it works pretty well.

    String path = "file.txt";
    ByteArrayOutputStream byteBuffer = new ByteArrayOutputStream();
    int bufferSize = 1024;
    byte[] buffer = new byte[bufferSize];
    int len = 0;
    while ((len = inputStream.read(buffer)) != -1) {
        byteBuffer.write(buffer, 0, len);
    }
    FileOutputStream stream = new FileOutputStream(path); 
    stream.write(byteBuffer.toByteArray());

My concern i that the result from inputstream could potentially be larger than the heap size in android and I could get OutOfMemory exceptions if the entire byte array is in memory. What is the most elegant way to write the inputStream to file in chunks, such that the byte array is never larger than the heap size?

like image 971
JoeLallouz Avatar asked Mar 28 '12 15:03

JoeLallouz


People also ask

Is it possible to create a File object from InputStream?

Since Java 7, you can do it in one line even without using any external libraries: Files. copy(inputStream, outputPath, StandardCopyOption. REPLACE_EXISTING);

Can we convert InputStream to File in Java?

Using nio packages exposed by Java 8, you can write an InputStream to a File using Files. copy() utility method.


2 Answers

Don't write to the ByteArrayOutputStream. Write directly to the FileOutputStream.

String path = "file.txt";
FileOutputStream output = new FileOutputStream(path); 
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = inputStream.read(buffer)) != -1) {
    output.write(buffer, 0, len);
}
like image 98
Matt Ball Avatar answered Oct 09 '22 10:10

Matt Ball


I went with the advice to skip the ByteArrayOutputStream and write to the FileOutputStream and this seems to address my concerns. With one quick adjustment, where the FileOutputStream is decorated by a BufferedOutputStream

String path = "file.txt";
OutputStream stream = new BufferedOutputStream(new FileOutputStream(path)); 
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = is.read(buffer)) != -1) {
    stream.write(buffer, 0, len);
}
if(stream!=null)
    stream.close();
like image 32
JoeLallouz Avatar answered Oct 09 '22 11:10

JoeLallouz