Android Image upload/download with Base64 into JSON causes Out of memory error

Question

I currently encode and decode images to Base64. I overcame the initial issue with OOM's with the use of streams to encode the images into strings.

My issue now is that I cannot fathom how to add multiple Base64 encoded strings for multiple resolutions images (5620 x 3747 - 4.92MB or 3264 x 1836 - 1.35MB) to a JSON Object via Gson. Currently Gson throws an OOM exception only with 2 Base64 Strings from a 5312 x 2988 - 4.95 MB Image.

I understand that android may only be able to spare 16/20Mb per application, so this conversion must be way over the limit.

How can I write the Base64 String in a stream to a JSON object that will contain the specific values needed to post into my server?

Would it be easier to change my server to accept a Multi-Part request instead of a JSON based POJO with multiple Base64 Strings? I currently use Volley and there isn't an official Multi-Part Request as well as IO streaming.

If it's a matter of compression, how much compression should I apply to the image before encoding into a Base64 String? I ideally want to lose barely any quality but have optimal compression levels.

Bit more Information

I am uploading multiple different resolution images as it is a test for compatibility. For example, all the images that I am sending up have been taken on low resolution and extremely high resolution devices as my App relies on these images for functionality. I am trying to prove that any image (to a certain extent, mainly images captured on mobile devices) can be handled by my application.

I understand that some images may be so large that by loading them into memory will cause exceptions. This is something I will try and handle later.

In some cases the images that will be uploaded can span from 1 to 200.

I'm trying to look for the most optimal solution that will scale well.

Answer 1

... for multiple resolutions images (5620 x 3747 - 4.92MB or 3264 x 1836 - 1.35MB)...

Not sure if this is the file size or the memory needed to allocate the image in memory, but taking a look at the following link: http://www.scantips.com/basics1d.html, I see this:

For a 4000 x 2500 pixel image,

then: 4000 x 2500 pixels = 4000x2500 = 10 megapixels

4000x2500 x 3 = 30 million bytes (if 24-bit RGB)

30,000,000 bytes / (1024 x 1024) = 28.61 megabytes (MB)

This is simply how large the data is - For ANY 24-bit 10 megapixel image, but JPG files compress it smaller (only while in the file).

I think that the images you're handling are taking much more memory than you expect.

Also, taking a look at this question and answer: https://stackoverflow.com/a/11402374/3393666, we know that a base64 representation of an image take up to 37% more memory than the original image size.

How can I write the Base64 String in a stream to a JSON object that will contain the specific values needed to post into my server?

I think you could do this (with small images not large ones) by simply adding the base64 representation of the image in a JSON object then posting it to the server.

Would it be easier to change my server to accept a Multi-Part request instead of a JSON based POJO with multiple Base64 Strings?

In my opinion that would be your best option to implement what you're trying to achieve.

I currently use Volley and there isn't an official Multi-Part Request as well as IO streaming.

You can take a look at this answer: https://stackoverflow.com/a/16803473/3393666, you can definitely can do it with volley, but if you want an alternative you can try with retrofit (http://square.github.io/retrofit/) they support Multipart our of the box.

eg:

@Multipart
@PUT("user/photo")
Call<User> updateUser(@Part("photo") RequestBody photo, @Part("description") RequestBody description); 

Answer 2

I looked into using Volley as a mechanism to transport large JSON objects to a server and found this answer. This answer essentially proved that Volley would be a bad idea for what I wanted.

I switched to OkHttp and now use their streaming methods allowing the JSON to be streamed to the server and then read the response with a streamlined approach. I used the GSON library to parse the response as OKHttp allows the response JSON/Object to be streamed into a reader object which Gson then uses for internal streaming and parsing to an POJO class.

The ony reason why I did not switch to a Multi-Part request was due to the server side implementation being rigid and unchangeable to cover for Multi-Part requests, it strictly expected a JSON representation of data and files.

For handling Base64 Images on Android I severely reccomend not using the String representation and merely converting to Bytes to save on using an excessive amount of memory. I read this article on String memory usage and management. With the Bytes you may easily transport the data without leaving a massive footprint on memory.

For Displaying the images I still avoid the bytes to String conversion by using the Image library Glide. They allow you to pass in a byte[] which was of massive convenience.