Android: OutOfMemoryError while uploading video - how best to chunk?

Question

I have the same problem as described here, but I will supply a few more details. While trying to upload a video in Android, I'm reading it into memory, and if the video is large I get an OutOfMemoryError.

Here's my code:

// get bytestream to upload
videoByteArray = getBytesFromFile(cR, fileUriString);

public static byte[] getBytesFromFile(ContentResolver cR, String fileUriString) throws IOException {
    Uri tempuri = Uri.parse(fileUriString);
    InputStream is = cR.openInputStream(tempuri);
    byte[] b3 = readBytes(is);
    is.close();
    return b3;
}
public static byte[] readBytes(InputStream inputStream) throws IOException {
    ByteArrayOutputStream byteBuffer = new ByteArrayOutputStream();
    // this is storage overwritten on each iteration with bytes
    int bufferSize = 1024;
    byte[] buffer = new byte[bufferSize];
    int len = 0;
    while ((len = inputStream.read(buffer)) != -1) {
        byteBuffer.write(buffer, 0, len);
    }
    return byteBuffer.toByteArray();
}

And here's the traceback (the error is thrown on the byteBuffer.write(buffer, 0, len) line):

04-08 11:56:20.456: ERROR/dalvikvm-heap(6088): Out of memory on a 16775184-byte allocation.
04-08 11:56:20.456: INFO/dalvikvm(6088): "IntentService[UploadService]" prio=5 tid=17 RUNNABLE
04-08 11:56:20.456: INFO/dalvikvm(6088):   | group="main" sCount=0 dsCount=0 s=N obj=0x449a3cf0 self=0x38d410
04-08 11:56:20.456: INFO/dalvikvm(6088):   | sysTid=6119 nice=0 sched=0/0 cgrp=default handle=4010416
04-08 11:56:20.456: INFO/dalvikvm(6088):   at java.io.ByteArrayOutputStream.expand(ByteArrayOutputStream.java:~93)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:218)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at com.android.election2010.UploadService.readBytes(UploadService.java:199)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at com.android.election2010.UploadService.getBytesFromFile(UploadService.java:182)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at com.android.election2010.UploadService.doUploadinBackground(UploadService.java:118)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at com.android.election2010.UploadService.onHandleIntent(UploadService.java:85)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:30)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at android.os.Handler.dispatchMessage(Handler.java:99)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at android.os.Looper.loop(Looper.java:123)
04-08 11:56:20.456: INFO/dalvikvm(6088):   at android.os.HandlerThread.run(HandlerThread.java:60)
04-08 11:56:20.467: WARN/dalvikvm(6088): threadid=17: thread exiting with uncaught exception (group=0x4001b180)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088): Uncaught handler: thread IntentService[UploadService] exiting due to uncaught exception
04-08 11:56:20.467: ERROR/AndroidRuntime(6088): java.lang.OutOfMemoryError
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at java.io.ByteArrayOutputStream.expand(ByteArrayOutputStream.java:93)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:218)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at com.android.election2010.UploadService.readBytes(UploadService.java:199)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at com.android.election2010.UploadService.getBytesFromFile(UploadService.java:182)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at com.android.election2010.UploadService.doUploadinBackground(UploadService.java:118)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at com.android.election2010.UploadService.onHandleIntent(UploadService.java:85)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:30)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at android.os.Handler.dispatchMessage(Handler.java:99)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at android.os.Looper.loop(Looper.java:123)
04-08 11:56:20.467: ERROR/AndroidRuntime(6088):     at android.os.HandlerThread.run(HandlerThread.java:60)
04-08 11:56:20.496: INFO/Process(4657): Sending signal. PID: 6088 SIG: 3

I guess that as @DroidIn suggests, I need to upload it in chunks. But (newbie question alert) does that mean that I should make multiple PostMethod requests, and glue the file together at the server end? Or can I load the bytestream into memory in chunks, and glue it together in the Android code?

If anyone could give me a clue as to the best approach, I would be very grateful.

Answer 1

Jim Blackler is absolutely correct. You can't have complete video in memory in byte[]. So you should read file not to byte[] but directly to HttpUrlConnectio.getOutputStream().

But even after that you'll still get OutOfMemory because HttpUrlConnection caches all output data in memory and sends it to network only when you're complete writing.

To override this behaviour you can use HttpUrlConnection.setChunkedStreamingMode() or HttpUrlConnection.setFixedLengthStreamingMode(). If your server accepts stream chunks you can use setChunkedStreamingMode(). Otherwise you'll have to use setFixedLengthStreamingMode. To use setFixedLengthStreamingMode you'll need to know exact content length before starting streaming data, so it's a little bit tricky.

No more OutOfMemory at this point.

If you have your video in file you can choose another approach, much more simple. Use HttpClient and FileBody like that How to send HTTP POST request and receive response?. I think they implement all the methods above and you'll not have to worry about that.

Answer 2

You don't need to get the entire file in memory, then upload it on one go. Call connection.getOutputStream() and write() the bytes in chunks.

Answer 3

Just a little note :

The setChunkedStreamingMode and setFixedeLengthStreamingMode do NOT work in android 2.1, They work perfect in 2.0 and 2.2