Java HTTP process for 'streaming' video file

Question

I am creating a Java application which 'streams' a video file over http to a browser (currently Chrome v24.x). This video is sent to FFmpeg and the output of this is sent over HTTP.

Now, once the file is completely encoded the file is served using chunked transfer, and responding to range requests. Example headers:

Request

GET /file/9fe6b502-c127-47c2-b6d2-83ea58676a8d HTTP/1.1 : Host: localhost:1234 : Connection: keep-alive : Accept-Encoding: identity;q=1, *;q=0 : User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17 : Accept: */* : Referer: http://localhost:1234/media/9fe6b502-c127-47c2-b6d2-83ea58676a8d : Accept-Language: en-US,en;q=0.8 : Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 : Cookie: plushContainerWidth=100%25; plushNoTopMenu=0 : Range: bytes=0- :  

Response

HTTP/1.1 206 Partial Content : Connection: close : Date: Mon, 28 Jan 2013 14:51:52 GMT : Content-Type: video/mp4 : Etag: "9fe6b502-c127-47c2-b6d2-83ea58676a8d" : Accept-Ranges: bytes : Content-Range: bytes 0-625825/625826 : Content-Length: 625826 : Transfer-Encoding: chunked :  

Data sent is 625826 bytes, excluding header data and chunked overhead.

Now, this works just fine!

The trouble is when the GET request happens before the file encoding has completed. I have tried to start sending the file straight away, over HTTP, using just chunked transfer with no content length attributes or ranges because they are not currently known. This causes the browser to wait for the full file (and not start playing until the transfer has completed). In addition, when the file transfer is completed, the browser reports a video error that the file could not be played. Example headers:

Request

Request : GET /file/9fe6b502-c127-47c2-b6d2-83ea58676a8d HTTP/1.1 : Host: localhost:1234 : Connection: keep-alive : Accept-Encoding: identity;q=1, *;q=0 : User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17 : Accept: */* : Referer: http://localhost:1234/media/9fe6b502-c127-47c2-b6d2-83ea58676a8d : Accept-Language: en-US,en;q=0.8 : Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 : Cookie: plushContainerWidth=100%25; plushNoTopMenu=0 : Range: bytes=0- 

Response

HTTP/1.1 200 OK : Connection: close : Date: Mon, 28 Jan 2013 14:51:29 GMT : Content-Type: video/mp4 : Transfer-Encoding: chunked

Data sent is 625826 bytes, excluding header data and chunked overhead.

Does anyone have any ideas what's going wrong, or how to start playing a video without knowing the full length of the file?

Thanks for your time,

James.

//EDIT

As the request for the incomplete file states 'Range: bytes=0-' - can I reply with a partial content of 'n' bytes (say, 1000 bytes) with Content-Range: bytes 0-999/* ?

//EDIT 2 As requested, here is my code for outputting the file. This is condensed as the code actually spans several classes.

File f = new File(_filename);
RandomAccessFile raf = new RandomAccessFile(f, "r");
ChunkedOutputStream cos = new ChunkedOutputStream(_out, 1024 * 100);
byte[] bytes;

while (true){
    lBufferSizeMax = Math.min(lBufferSize, fc.length() - lCompleted);
    bytes = new byte[(int)lBufferSizeMax];

    lCurrentRead = fc.read(bytes);
    if (lCurrentRead == 0){
        break;
    }

    cos.write(bytes);
}

Answer 1

Be aware that ffmpeg is likely to rewrite portions of the file when encoding is ending. That means the metadata in the video file will be incomplete until the very end so there is no use trying to stream the beginning of the file before that time. You can check with replacing the output filename in your ffmpeg command line with a single dash and redirect stdout to a file. Instead of ffmpeg ... out.mov do ffmpeg ... - > out.mov. If you get an error message saying the format needs seekable output you are affected.

I have done almost exactly what you are trying to do, the results are in my Git repo. The container format I use, where I can play games with the metadata, is FLV. Unfortunately that is not what the iPad/iPhone devices out there like to see.