Java/Android HttpURLConnection setChunkedStreamingMode not working with all PHP servers

Question

I've spent the last day or so trying to debug this issue and I'm out of ideas. Basically I have an Android app that is POSTing some data to a PHP/Apache web server. This code seems to work fine when I point it at my local test server. It also seems to work fine when I point it at my production server, but ONLY when I comment out the line conn.setChunkedStreamingMode(maxBufferSize);. Once that line is enabled, the post only works on my local test server, but when posting to the production server, the PHP $_FILES array is empty. I've tried passing numerous values to setChunkedStreamingMode (including 0 and 1024) but none of these seem to fix the problem.

At this point I'm assuming the issue has to do with the way the production server's PHP is configured, but as far as I can tell, all the important parameters on the server are the same as on my test instance. Additionally, they're both running the same version of Apache and PHP. My production server is run by Bluehost.

Here's the Java code I'm using to upload:

HttpURLConnection conn = null;
DataOutputStream dos = null;
DataInputStream inStream = null;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "***************************************************";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 212144; // 1024*1024 = 1MB.  212144 is a quarter MB.
FileInputStream fileInputStream = null;
try
{
  // ------------------ CLIENT REQUEST
  fileInputStream = new FileInputStream(new File(existingFileWithFullPath));
  // open a URL connection to the Servlet
  URL url = new URL(BACKUP_POST_URL);
  // Open a HTTP connection to the URL
  conn = (HttpURLConnection) url.openConnection();
  // Allow Inputs
  conn.setDoInput(true);
  // Allow Outputs
  conn.setDoOutput(true);
  // Send in chunks (to avoid out of memory error)
  conn.setChunkedStreamingMode(maxBufferSize);
  // Don't use a cached copy.
  conn.setUseCaches(false);
  // Use a post method.
  conn.setRequestMethod("POST"); 
  conn.setRequestProperty("Connection", "Keep-Alive");
  conn.setRequestProperty("Content-Type", "multipart/form-data;boundary="
      + boundary);
  conn.setReadTimeout(200000); // 200 seconds...
  dos = new DataOutputStream(conn.getOutputStream());
  dos.writeBytes(twoHyphens + boundary + lineEnd);
  dos.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\""
      + fileName + "\"" + lineEnd);
  dos.writeBytes(lineEnd);
  // create a buffer of maximum size
  bytesAvailable = fileInputStream.available();
  bufferSize = Math.min(bytesAvailable, maxBufferSize);
  buffer = new byte[bufferSize];
  // read file and write it into form...
  bytesRead = fileInputStream.read(buffer, 0, bufferSize);
  while (bytesRead > 0)
  {
    try {
      dos.write(buffer, 0, bufferSize);          
    } catch (OutOfMemoryError oome) {
      Log.e(CommonStatic.LOG_NAME, "Out of memory error caught...");
      oome.printStackTrace();
      fileInputStream.close();
      throw new Exception("Out Of Memory!");
    }
    bytesAvailable = fileInputStream.available();
    bufferSize = Math.min(bytesAvailable, maxBufferSize);
    bytesRead = fileInputStream.read(buffer, 0, bufferSize);
  }
  // send multipart form data necesssary after file data...
  dos.writeBytes(lineEnd);
  dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
  fileInputStream.close();
  dos.flush();
  dos.close();

  // close streams
  Log.d(CommonStatic.LOG_NAME, "Backup file written to server successfully...");
}
catch (Exception ex)
{
  Log.e(CommonStatic.LOG_NAME, "Backup File Upload Error: " + ex.getMessage(), ex);
  throw new Exception (c.getString(R.string.SAVE_TO_CLOUD_ERROR));
}

And here's the PHP code I'm using on the other end to recieve:

$target = "userfiles/"; 
$target = $target . basename( $_FILES['uploadedfile']['name']);

if(move_uploaded_file($_FILES['uploadedfile']['tmp_name'], $target)) 
{ 
  echo "SUCCESS";
} 
else 
{ 
  echo "FAIL";
}

I stuck a print_r($_FILES); at the very beginning of the script to determine that $_FILES is empty on the production instance, but not on the test instance. Any ideas would be greatly appreciated.

Answer 1

Unfortunately I wasn't able to find a direct solution to this problem, but I was able to find a workaround. Instead of using conn.setChunkedStreamingMode I used conn.setFixedLengthStreamingMode, which I had tried before with no success. The key to getting conn.setFixedLengthStreamingMode to work is to pass it the full length of the data you are attempting to send (in this case the file), plus the length of the headers. Fortunately header length is usually fixed in code such as this, so once you figure out how big your header is (keeping in mind things like the filename, which gets sent in the header under Content-Disposition is also variable) you can just put it in as a fixed number. To figure out my header length, I ran the code first without specifying any length for the header. The error message I received back gave me an expected and an actual value for the number of bytes sent, which allowed me to calculate the header length.