Upload files by post to server OutOfMemory

Question

I'm developing a remote backup app, and Sometimes I need upload big files as for example 15 MB, I have tested in some phones I get an out of memory error

Is there a way to use less memory using this function?

public int uploadFile(String sourceFileUri) {


      String fileName = sourceFileUri;

      HttpURLConnection conn = null;
      DataOutputStream dos = null;  
      String lineEnd = "\r\n";
      String twoHyphens = "--";
      String boundary = "*****";
      int bytesRead, bytesAvailable, bufferSize;
      byte[] buffer;
      int maxBufferSize = 1 * 1024 * 1024; 
      File sourceFile = new File(sourceFileUri); 

      if (!sourceFile.isFile()) {

           runOnUiThread(new Runnable() {
               public void run() {
                /*   messageText.setText("Source File not exist :"
                           +uploadFilePath + "" + uploadFileName);*/
               }
           }); 

           return 0;

      }
      else
      {
           try { 

                 // open a URL connection to the Servlet
               FileInputStream fileInputStream = new FileInputStream(sourceFile);
               URL url = new URL(upLoadServerUri);

               // Open a HTTP  connection to  the URL
               conn = (HttpURLConnection) url.openConnection(); 
               conn.setDoInput(true); // Allow Inputs
               conn.setDoOutput(true); // Allow Outputs
               conn.setUseCaches(false); // Don't use a Cached Copy
               conn.setRequestMethod("POST");
               conn.setRequestProperty("Connection", "Keep-Alive");
               conn.setRequestProperty("ENCTYPE", "multipart/form-data");
               conn.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
               conn.setRequestProperty("uploaded_file", fileName); 

               dos = new DataOutputStream(conn.getOutputStream());

               dos.writeBytes(twoHyphens + boundary + lineEnd); 
               dos.writeBytes("Content-Disposition: form-data; name=\"uploaded_file\";filename=\""
                                         + fileName + "\"" + lineEnd);

               dos.writeBytes(lineEnd);

               // create a buffer of  maximum size
               bytesAvailable = fileInputStream.available(); 

               bufferSize = Math.min(bytesAvailable, maxBufferSize);
               buffer = new byte[bufferSize];

               // read file and write it into form...
               bytesRead = fileInputStream.read(buffer, 0, bufferSize);  

               while (bytesRead > 0) {

                 dos.write(buffer, 0, bufferSize);
                 bytesAvailable = fileInputStream.available();
                 bufferSize = Math.min(bytesAvailable, maxBufferSize);
                 bytesRead = fileInputStream.read(buffer, 0, bufferSize);   

                }

               // send multipart form data necesssary after file data...
               dos.writeBytes(lineEnd);
               dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);

               // Responses from the server (code and message)
               serverResponseCode = conn.getResponseCode();
              // String serverResponseMessage = conn.getResponseMessage();

              // Log.i("uploadFile", "HTTP Response is : " 
                //     + serverResponseMessage + ": " + serverResponseCode);

               if(serverResponseCode == 200){

                   runOnUiThread(new Runnable() {
                        public void run() {


                            /*messageText.setText(msg);
                            Toast.makeText(UploadToServer.this, "File Upload Complete.", 
                                         Toast.LENGTH_SHORT).show();*/
                        }
                    });                
               }    

               //close the streams //
               fileInputStream.close();
               dos.flush();
               dos.close();

          } catch (MalformedURLException ex) {

            //  ex.printStackTrace();

              runOnUiThread(new Runnable() {
                  public void run() {
                      /*messageText.setText("MalformedURLException Exception : check script url.");*/
                      Toast.makeText(UploadToServer.this, "MalformedURLException", Toast.LENGTH_SHORT).show();
                  }
              });

              Log.e("Upload file to server", "error: " + ex.getMessage(), ex);  
          } catch (Exception e) {

              //e.printStackTrace();

              runOnUiThread(new Runnable() {
                  public void run() {
                      //messageText.setText("Got Exception : see logcat ");
                      Toast.makeText(UploadToServer.this, "Got Exception : see logcat ", 
                              Toast.LENGTH_SHORT).show();
                  }
              });
              Log.e("Upload file to server Exception", "Exception : " 
                                               + e.getMessage(), e);  
          }
         // dialog.dismiss();       
          return serverResponseCode; 

       } // End else block 
     } 

Here the error log

java.lang.OutOfMemoryError
at java.io.ByteArrayOutputStream.expand(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:218)
at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl$DefaultHttpOutputStream.write(HttpURLConnectionImpl.java:750)
at java.io.DataOutputStream.write(DataOutputStream.java:101)
at com.androidexample.uploadtoserver.UploadToServer.a(SourceFile:151)
at com.androidexample.uploadtoserver.e.run(SourceFile:62)
at java.lang.Thread.run(Thread.java:1102)

After modify and add Fixed no Buffer i get this error

Exception : expected 589715 bytes but received 589840
java.io.IOException: expected 589715 bytes but received 589840
at libcore.net.http.FixedLengthOutputStream.write(FixedLengthOutputStream.java:39)
at java.io.DataOutputStream.write(DataOutputStream.java:98)
at com.androidexample.uploadtoserver.UploadToServer.uploadFile(UploadToServer.java:152)
at com.androidexample.uploadtoserver.UploadToServer$1.run(UploadToServer.java:62)
at java.lang.Thread.run(Thread.java:856)

I guess this bytes of difference are the headers data right? how can I get the length() of the headers?

Answer 1

You should use either the setChunkedStreamingMode() or setFixedLengthStreamingMode() methods of the HttpURLConnection. This will prevent the buffering of your data in memory and exhausting it.

Relevant quote from the documentation:

For best performance, you should call either setFixedLengthStreamingMode(int) when the body length is known in advance, or setChunkedStreamingMode(int) when it is not. Otherwise HttpURLConnection will be forced to buffer the complete request body in memory before it is transmitted, wasting (and possibly exhausting) heap and increasing latency.

More here: http://developer.android.com/reference/java/net/HttpURLConnection.html

Answer 2

Add

conn.setFixedLengthStreamingMode(sourceFile.length());

below:

conn = (HttpURLConnection) url.openConnection();