SOAP - Very large XML response - OutOfMemoryError

Question

First, this question looks like Very large SOAP response - Android- out of memory error subject. Because of my English weakness and the similarity of this problem, with a view to facilitate understanding some statement passages were copied.

I have an application where i need to download a large amount of data via a SOAP call to a webservice. The response is then sent to a function which display the XML file.

The data is more than 11MB in size and i have a java.lang.OutOfMemoryError everytime.

Modifying the webservice to give out smaller amounts of data is not an option.

I use "Http request" to get datas. I know that : my resquest is fine, soapUi and wireshark return expected responses.

But my AVD isn't able to pass this line

HttpResponse httpResponse = httpClient.execute(httpPost);

After some minutes of work (during which wireshark recovers some expected queries) this error is made

 Out of memory on a 16705124-byte allocation.

I tried to upgrade the SD card size to 20GB, yet, error still.

Parsing httpResponse is probably the next step, is it possible to parse HttpResponse while it receives data by fragmenting it into several parts by exmple?

Do you have an idea ?

Thanks, Dsandre

Answer 1

Thank's Graeme, opening connection as a byte stream works. If it could be helpfull to somebody, this is my source code

        //Make a temporary file to save data
        File responseFile = File.createTempFile("SOAP", "xml", context.getFilesDir());      
        int nbCharRead = 0; int i=0; int totalRead = 0;

        //Send query
        OutputStream outputS = url_Connection.getOutputStream();
        Writer w_out = new OutputStreamWriter(outputS);
        w_out.write(webServiceXml);
        w_out.flush();
        w_out.close();


        //Buffers
        BufferedReader bufReader = new BufferedReader(new InputStreamReader(url_Connection.getInputStream()));
        BufferedWriter bufWriter = new BufferedWriter(new FileWriter(responseFile));
        char[] buffer = new char[10000];



        while((nbCharRead = bufReader.read(buffer, 0, 10000)) != -1)
        {
            totalRead += nbCharRead;
            Log.d("Test InputStream", "i: " + i++ +" - " + nbCharRead + " -> " + totalRead);
            bufWriter.write(buffer, 0, nbCharRead );
        }       

        if(bufWriter != null)
        {
            bufWriter.flush();
            bufWriter.close();
        }


        Log.w(MsgLog, "--- Stream Got--- ; Total : " + totalRead);

Answer 2

You can open a connection as a byte stream:

URL url = new URL("www.google.com");
InputStream in = new BufferedInputStream(url.openStream());
byte[] buffer = new byte[1024];
int chunk = 0;
while ((chunk =in.read(buffer ))!=-1)
{
    proccessChunk(chunk);
}
in.close();

From this point you should be able to metre the response and process it bit-by-bit, saving each processed chunk to text files (or, if it is relational data to SQLite).

HTH