Very large SOAP response - Android- out of memory error

Question

I have an application where i need to download a large amount of data via a SOAP call to a webservice into the application when it is first run. The response is then sent to a function which converts the XML and stores the data in a db file.

The data is more than 16MB in size and i have a java.lang.OutOfMemoryError everytime.

Modifying the webservice to give out smaller amounts of data is not an option.

Is there a way to be able to download the large data? Something like an InputStream perhaps?

This is my code

public Protocol[] getProtocols() {

    String METHOD_NAME = "GetProtocols";
    String SOAP_ACTION = "urn:protocolpedia#GetProtocols";
    Log.d("service", "getProtocols");
    SoapObject response = invokeMethod(METHOD_NAME, SOAP_ACTION);
    return retrieveProtocolsFromSoap(response);
}

private SoapObject invokeMethod(String methodName, String soapAction) {
    Log.d(TAG, "invokeMethod");
    SoapObject request = GetSoapObject(methodName);
    SoapSerializationEnvelope envelope = getEnvelope(request);
    return makeCall(envelope, methodName, soapAction);

}

Can anyone suggest what should be done in this case?

Thanks and regards Mukul

Answer 1

Just an update, I found that the "call" method in AndroidHttpTransport was running out of memory at this line -

           if (debug) {
                    ByteArrayOutputStream bos = new ByteArrayOutputStream();
                    byte[] buf = new byte[256];
                    while (true) {
                        int rd = is.read(buf, 0, 256);
                        if (rd == -1)
                            break;
                        bos.write(buf, 0, rd);
                    }
                    bos.flush();
                    buf = bos.toByteArray(); //Goes out of memory here
                    responseDump = new String(buf);
                    is.close();
                    is = new ByteArrayInputStream(buf);

the call to toByteArray takes a lot of memory, so to overcome this, instead of converting the response to a byte array, i now directly write it to an XML file, and this is saved at a location of my choice. Here -

if (debug) {
    FileOutputStream bos = new FileOutputStream("/data/data/com.mypackage.myapp/response.xml");
    byte[] buf = new byte[1048576];
    int current = 0; int i=0; int newCurrent = 0;
    while ((current = inputStream.read(buf)) != -1) {
        newCurrent = newCurrent + current;
    Log.d("current", "Current = " + current + " total = "+newCurrent+" i = "+i++);
                    bos.write(buf, 0, current);
                }
                bos.flush();
}

The device no longer runs out of memory, and i have a custom parse method that takes this XML and writes it to the DB.