Node.js / Express video streaming (HTTP 206 Partial Content)

Question

I have a binary document (mp4 video file) in a database (MarkLogic). I am using the database's Node.js API to stream the document in chunks. The setup looks like this:

html file

<video controls="controls" width="600">
  <source src="/video/myvideo.mp4" type="video/mp4">
</video>

In express I have then setup a route which handles the /video/:param route (in the database the video has the unique identifier which is the string '/video/myvideo.mp4')

node.js

// I'm only showing the relevant things in here

const serveVideo = (req, res) => {
  var stream = db.documents.read('/gopro/malta.mp4').stream('chunked');

  var chunks = [];
  var chunkBytes = 0;
  var start = 0;
  stream.on('data', (chunk) => {
    var headers;
    var range = req.headers.range;
    var total = 214335483; //total length of vid in bytes

    if (range) {
      var chunkSize = chunk.length;
      // (start === 0) ? start = 0 : start += chunkBytes;
      if (chunkBytes === 0) {
        start = 0
      } else {
        start = chunkBytes + 1
      }
      chunkBytes += chunkSize;

      headers = {
        'Content-Range': 'bytes ' + start + '-' + chunkBytes + '/' + total,
        'Accept-Ranges': 'bytes',
        'Content-Length': chunkSize,
        'Content-Type': 'video/mp4'
      };
      res.writeHead(206, headers);
      chunks.push(chunk);
    }
  });
  stream.on('end', () => {
    var allChunks = Buffer.concat(chunks);
    res.end(allChunks);
  });
});


router.route('/video/:uri').get(serveVideo);

Now of course the above fails with 'Error: Can't set headers after they are sent.' which is all fair and square. But I can't get my head around this - the .stream('chunked') call forces the database to retrieve the document in chunks and I do see those chunks just fine, however how can I return a 206 for the browser? I can't do it in the .on('data') as data is being streamed so the header would be sent multiple times. I guess which database I'm using is not really relevant - I would like to understand the concept, or at least see what I'm doing wrong.

Any help is appreciated. All the examples and other discussions that I have seen that stream video using Node.js are reading the video file from disk.

update

Making a change to the code now allows FF to play the video but not Chrome:

let stream = db.documents.read({uris:'/gopro/malta.mp4'}).stream('chunked');
stream.pipe(res);

There are no errors in Chrome's console. Here are the header details - note that there are two requests for the mp4 file:

1st

Response Headers
Connection:keep-alive
Date:Sat, 21 May 2016 17:05:30 GMT
Transfer-Encoding:chunked
X-Powered-By:Express

Request Headers
view source
Accept:*/*
Accept-Encoding:identity;q=1, *;q=0
Accept-Language:en-US,en;q=0.8,hu;q=0.6,ro;q=0.4,it;q=0.2
Cache-Control:no-cache
Connection:keep-alive
Cookie:__distillery=v20150227_a8e22306-65b3-4c2e-9a8a-159e308156ad; __smToken=7nYU8NYQY15mPowjjCZsS5D3
DNT:1
Host:localhost:8080
Pragma:no-cache
Range:bytes=0-
Referer:http://localhost:8080/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36

2nd

Response Headers
Connection:keep-alive
Date:Sat, 21 May 2016 17:05:31 GMT
Transfer-Encoding:chunked
X-Powered-By:Express

Request Headers
view source
Accept:*/*
Accept-Encoding:identity;q=1, *;q=0
Accept-Language:en-US,en;q=0.8,hu;q=0.6,ro;q=0.4,it;q=0.2
Cache-Control:no-cache
Connection:keep-alive
Cookie:__distillery=v20150227_a8e22306-65b3-4c2e-9a8a-159e308156ad; __smToken=7nYU8NYQY15mPowjjCZsS5D3
DNT:1
Host:localhost:8080
Pragma:no-cache
Range:bytes=28-
Referer:http://localhost:8080/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36

Answer 1

The on('data') callback could always have a closure over an isFirstChunk variable that is initialized to true and have a test that, if isFirstChunk is true, emits the headers and sets isFirstChunk to false.

It would be preferable to pipe the stream if possible. npm might offer a stream library (maybe even through2()?) that has an event when the first data arrives.

For the longer term, you could reasonably file an RFE for an event at the start of a stream.

Hoping that helps,

Answer 2

You most likely don't need all the chunk stuff. Set headers manually:

res.status(206);

Then simply pipe the response:

let stream = db.yourChunkStuff();
stream.pipe(res);

The simplest way to is pipe streams.