How to post large video on a server, in Android?

Question

I am trying to post a large video (nearly 1 GB).

I am using FTP to send video to a server, but the upload stops after a while. On the server the video crashes, but I am able to upload a smaller sized video.

I've also used HTTP to send video to the server, sent as a Base64 enoded string, but there is an out of memory exception while encoding.

I've tried to upload the video as a file, but without success. What is the best way to upload a large video to a server?

Answer 1

Use HTTP POST, and post content as Form-based File Upload (mime type: multipart/form-data). This system is standard on web for sending forms and/or uploading files.

Use HTTP chunked post mode, so that size doesn't need to be known beforehand, and you can stream any file in small parts. You still have to make some code on server (e.g. in PHP) to accept the file and do what is needed.

Use HttpURLConnection to initiate connection. Then use my attached class to send the data. It will create proper headers, etc, and you'll use it as OutputStream to write your raw data to it, then call close, and you're done. You can overrite its onHandleResult to handle resulting error code.

public class FormDataWriter extends FilterOutputStream{
   private final HttpURLConnection con;

   /**
    * @param formName name of form in which data are sent
    * @param fileName
    * @param fileSize   size of file, or -1 to use chunked encoding
    */
   FormDataWriter(HttpURLConnection con, String formName, String fileName, long fileSize) throws IOException{
      super(null);
      this.con = con;
      con.setDoOutput(true);

      String boundary = generateBoundary();
      con.setRequestProperty(HTTP.CONTENT_TYPE, "multipart/form-data; charset=UTF-8; boundary="+boundary);
      {
         StringBuilder sb = new StringBuilder();
         writePartHeader(boundary, formName, fileName==null ? null : "filename=\""+fileName+"\"",
               "application/octet-stream", sb);
         headerBytes = sb.toString().getBytes("UTF-8");

         sb = new StringBuilder();
         sb.append("\r\n");
         sb.append("--"+boundary+"--\r\n");
         footerBytes = sb.toString().getBytes();
      }
      if(fileSize!=-1) {
         fileSize += headerBytes.length + footerBytes.length;
         con.setFixedLengthStreamingMode((int)fileSize);
      }else
         con.setChunkedStreamingMode(0x4000);
      out = con.getOutputStream();
   }

   private byte[] headerBytes, footerBytes;

   private String generateBoundary() {
      StringBuilder sb = new StringBuilder();
      Random rand = new Random();
      int count = rand.nextInt(11) + 30;
      int N = 10+26+26;
      for(int i=0; i<count; i++) {
         int r = rand.nextInt(N);
         sb.append((char)(r<10 ? '0'+r : r<36 ? 'a'+r-10 : 'A'+r-36));
      }
      return sb.toString();
   }

   private void writePartHeader(String boundary, String name, String extraContentDispositions, String contentType, StringBuilder sb) {
      sb.append("--"+boundary+"\r\n");
      sb.append("Content-Disposition: form-data; charset=UTF-8; name=\""+name+"\"");
      if(extraContentDispositions!=null)
         sb.append("; ").append(extraContentDispositions);
      sb.append("\r\n");
      if(contentType!=null)
         sb.append("Content-Type: "+contentType+"\r\n");
      sb.append("\r\n");
   }

   @Override
   public void write(byte[] buffer, int offset, int length) throws IOException{
      if(headerBytes!=null) {
         out.write(headerBytes);
         headerBytes = null;
      }
      out.write(buffer, offset, length);
   }

   @Override
   public void close() throws IOException{
      flush();
      if(footerBytes!=null) {
         out.write(footerBytes);
         footerBytes = null;
      }
      super.close();
      int code = con.getResponseCode();
      onHandleResult(code);
   }
   protected void onHandleResult(int code) throws IOException{
      if(code!=200 && code!=201)
         throw new IOException("Upload error code: "+code);
   }
}

Answer 2

I guess it failed because of a timeout by the big size.

Since

Small size video uploaded successfully

, My suggestion is

1. Split one big file to several small files.
2. Upload one by one or several together based on the condition of network.
3. Join all of parts (after all of those uploaded successfully) at server.
4. Because of small size, Re-upload failed part will be easy.

Just a theroy.

This site may help .

---

Added 08.01.2013

It has been a while, don't know if you still need this. Anyway, I wrote some simple codes implement the theory above, because of interest mainly.

1. Split one big file to several small files. Read the big file into several small parts.

   ByteBuffer bb = ByteBuffer.allocate(partSize);
   int bytesRead = fc.read(bb);
   if (bytesRead == -1) {
       break;
   }
   byte[] bytes = bb.array();
   parts.put(new Part(createFileName(fileName, i), bytes));
   

2. Upload one by one or several together based on the condition of network.

   Part part = parts.take();
   if (part == Part.NULL) {
       parts.add(Part.NULL);// notify others to stop.
       break;
   } else {
       uploader.upload(part);
   }
   

3. Join all of parts (after all of those uploaded successfully) at server. Because it is via HTTP, so it can be in any language, such as Java, PHP, Python, etc. Here is a java example.

   ...
   try (FileOutputStream dest = new FileOutputStream(destFile, true)) {
   
       FileChannel dc = dest.getChannel();// the final big file.
       for (long i = 0; i < count; i++) {
           File partFile = new File(destFileName + "." + i);// every small parts.
           if (!partFile.exists()) {
               break;
           }
           try (FileInputStream part = new FileInputStream(partFile)) {
               FileChannel pc = part.getChannel();
               pc.transferTo(0, pc.size(), dc);// combine.
           }
           partFile.delete();
       }
       statusCode = OK;// set ok at last.
   } catch (Exception e) {
       log.error("combine failed.", e);
   }
   

I put all codes on GitHub. And made a Android example too.

Please have a look if you still need.