Android httpclient file upload data corruption and timeout issues

Question

i'm having issues with uploading images in android.

i'm using apache httpmime 4.1 lib the code is like this:

MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);

reqEntity.addPart("image", new FileBody(new File(AndorraApplication.getPhotosPath() + "/" + entity.getFileName()), "image/jpeg"));
resp = NetworkUtils.sendHttpRequestMultipart(EXPORT_PHOTOS_URI, reqEntity);

NetworkUtils class:

public class NetworkUtils {
    public static final int REGISTRATION_TIMEOUT = 3 * 1000; 
    public static final int WAIT_TIMEOUT = 5 * 1000;

    public static HttpResponse sendHttpRequestMultipart(String uri, MultipartEntity entity) {
        HttpClient mHttpClient = new DefaultHttpClient();
        final HttpParams params = mHttpClient.getParams();
        HttpConnectionParams.setConnectionTimeout(params, REGISTRATION_TIMEOUT);
        HttpConnectionParams.setSoTimeout(params, WAIT_TIMEOUT);
        ConnManagerParams.setTimeout(params, WAIT_TIMEOUT);

        HttpPost post = new HttpPost(uri);
        post.addHeader(entity.getContentType());
        post.setEntity(entity);
        HttpResponse resp = mHttpClient.execute(post);
    }
}

sometimes everything works fine, but sometimes (especially on a slow connection) the image is uploaded very corrupted. the example is here: http://pixelbirthcloud.com/574_orig.jpg

it doesn't throw any exceptions. the lenght of the uploaded file is same as the original one.. tried to change mime type to application/octet-stream or remove it at all. tries playing with timeouts. still same result. end users upload corrupted images almost all the time (although i managed to get bronem images just 2 times only).. the image size was 2.5 megs at first, but then i reduced it to 500-700 kb. didn't fix the problem though.

haven't tried to change the apache's library.. maybe it is the problem.. but as far as i read the net, nobody was experiencing this with httpmime library.

what can it be? i'm completely lost now :(

the other issue is that timeouts sometimes not working.

like when it comes to this line: HttpResponse resp = mHttpClient.execute(post); and i disable 3g connection it just waits for like 17-20 minutes instead of 3 or 5 seconds.. and only then throws exception. tried different methods. like this:

        HttpParams params = new BasicHttpParams();
        HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
        HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);
        HttpProtocolParams.setUseExpectContinue(params, false);  
        HttpConnectionParams.setConnectionTimeout(params, 10000);
        HttpConnectionParams.setSoTimeout(params, 10000);
        ConnManagerParams.setMaxTotalConnections(params, 5);
        ConnManagerParams.setTimeout(params, 30000);

        SchemeRegistry registry = new SchemeRegistry();
        registry.register(new Scheme("http",PlainSocketFactory.getSocketFactory(), 80));
        registry.register(new Scheme("https",PlainSocketFactory.getSocketFactory(), 80));
        ThreadSafeClientConnManager manager = new ThreadSafeClientConnManager(params, registry);
        HttpClient httpclient = new DefaultHttpClient(manager, params);

but still doesn't work :)

Answer 1

See My Code of Image Uploader and it worked great for me This class Uploads a file to the server plus in the end read the XML reply also. Filter the code as per your requirement.. It worked pretty smooth for me

package com.classifieds;

import java.io.DataOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;

import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;

import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;

import org.xml.sax.Attributes;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import org.xml.sax.XMLReader;
import org.xml.sax.helpers.DefaultHandler;


import android.util.Log;

public class Uploader 
{

    private String Tag = "UPLOADER";
    private String urlString ;//= "YOUR_ONLINE_PHP";
    HttpURLConnection conn;
    String exsistingFileName ;

    private void uploadImageData(String existingFileName , String urlString)
    {
        String lineEnd = "\r\n";
        String twoHyphens = "--";
        String boundary = "*****";
        try {
            // ------------------ CLIENT REQUEST

            Log.e(Tag, "Inside second Method");

            FileInputStream fileInputStream = new FileInputStream(new File(
                    exsistingFileName));

            // open a URL connection to the Servlet

            URL url = new URL(urlString);

            // Open a HTTP connection to the URL

            conn = (HttpURLConnection) url.openConnection();

            // Allow Inputs
            conn.setDoInput(true);

            // Allow Outputs
            conn.setDoOutput(true);

            // Don't use a cached copy.
            conn.setUseCaches(false);

            // Use a post method.
            conn.setRequestMethod("POST");

            conn.setRequestProperty("Connection", "Keep-Alive");

            conn.setRequestProperty("Content-Type",
                    "multipart/form-data;boundary=" + boundary);

            DataOutputStream dos = new DataOutputStream(conn.getOutputStream());

            dos.writeBytes(twoHyphens + boundary + lineEnd);
            dos
                    .writeBytes("Content-Disposition: post-data; name=uploadedfile;filename="
                            + exsistingFileName + "" + lineEnd);
            dos.writeBytes(lineEnd);

            Log.v(Tag, "Headers are written");

            // create a buffer of maximum size

            int bytesAvailable = fileInputStream.available();
            int maxBufferSize = 1000;
            // int bufferSize = Math.min(bytesAvailable, maxBufferSize);
            byte[] buffer = new byte[bytesAvailable];

            // read file and write it into form...

            int bytesRead = fileInputStream.read(buffer, 0, bytesAvailable);

            while (bytesRead > 0) {
                dos.write(buffer, 0, bytesAvailable);
                bytesAvailable = fileInputStream.available();
                bytesAvailable = Math.min(bytesAvailable, maxBufferSize);
                bytesRead = fileInputStream.read(buffer, 0, bytesAvailable);
            }

            // send multipart form data necesssary after file data...

            dos.writeBytes(lineEnd);
            dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);

            // close streams
            Log.v(Tag, "File is written");
            fileInputStream.close();
            dos.flush();
            dos.close();

        } catch (MalformedURLException ex) {
            Log.e(Tag, "error: " + ex.getMessage(), ex);
        }

        catch (IOException ioe) {
            Log.e(Tag, "error: " + ioe.getMessage(), ioe);
        }


        SAXParserFactory spf = SAXParserFactory.newInstance();
        SAXParser sp = null;
        try {
            sp = spf.newSAXParser();
        } catch (ParserConfigurationException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (SAXException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        // Get the XMLReader of the SAXParser we created.
        XMLReader xr = null;
        try {
            xr = sp.getXMLReader();
        } catch (SAXException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        // Create a new ContentHandler and apply it to the XML-Reader
        MyExampleHandler1 myExampleHandler = new MyExampleHandler1();
        xr.setContentHandler(myExampleHandler);

        // Parse the xml-data from our URL. 
        try {
            xr.parse(new InputSource(conn.getInputStream()));
        //xr.parse(new InputSource(new java.io.FileInputStream(new java.io.File("login.xml")))); 
        } catch (MalformedURLException e) {
            Log.d("Net Disconnected", "NetDisconeeted");
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            Log.d("Net Disconnected", "NetDisconeeted");
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (SAXException e) {
            Log.d("Net Disconnected", "NetDisconeeted");
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        // Parsing has finished.

    }

    public Uploader(String existingFileName, boolean isImageUploading , String urlString ) {

        this.exsistingFileName = existingFileName;
        this.urlString = urlString;

    }

    class MyExampleHandler1 extends DefaultHandler
    {
    // ===========================================================
    // Methods
    // ===========================================================

    @Override
    public void startDocument() throws SAXException {

    }

    @Override
    public void endDocument() throws SAXException {
        // Nothing to do
    }

    @Override
    public void startElement(String namespaceURI, String localName,
              String qName, Attributes atts) throws SAXException {


    }

    /** Gets be called on closing tags like:
     * </tag> */
    @Override
    public void endElement(String namespaceURI, String localName, String qName)
              throws SAXException {


    }

    /** Gets be called on the following structure:
     * <tag>characters</tag> */
    @Override
    public void characters(char ch[], int start, int length) {

     }
    }

}

Answer 2

I had the same corruption problem on 80% of my uploaded files. Emulator didn't fail to upload though. Corrupted files were 1k larger that original ones. Then I set the buffer of the output stream to 1 byte and it began to work with no problem. Finally I let it be 8 bytes and haven't had no more corruption problems. A buffer of about 80 or 50, I can't remember, also failed. Don't understand what the problem is but I'm happy it is working this way. This page was so inspiring thanks.