How to create a frame-accurate video sequencer with HTML5 <video>?

Question

I am attempting to build a video sequencer that is capable of playing back a list of video URLs seamlessly in series. There cannot be a gap between the videos; the playback needs to be as close to frame-accurate as possible.

Attempt #1 I used a fairly obvious and straightforward approach, similar to that mentioned in this thread. HTML 5 video or audio playlist

The issue with this approach was that each time one video ended and the subsequent video was specified as the new source, that new video still needed to load, resulting in a potentially long gap. If it is possible to force all the videos to preload even before they are named in video.src, this approach could still potentially work.

Attempt #2 I rewrote the script so each video in the list would result in a separate video element being created, but not attached to the document (using createElement). Then as each video ended, I removed the previous node and appended the next element.

Code executed on page load:

for (i in timelinePlay)
    if (timelinePlay[i] != null)        
    {
        var element = document.createElement('video');
        element.id = 'video1-' + (i);
        element.src = timelinePlay[i]['URL'];
        element.preload = 'load';
        video1Array[i] = element;
    }

Code for video 'ended' event:

videoElement.addEventListener("ended", function (event) {
    if (this.currentTime > 0)
    {
        if (player.hasChildNodes())
            while (player.childNodes.length > 0)
                player.removeChild(player.firstChild);

        player = document.getElementById('canvas-player');
        player.appendChild(video1Array[timelineCurrent]);

        nextVideo = document.getElementById('video1-' + timelineCurrent);

        nextVideo.play();

        timelineCurrent++;
    }
}, false);

(Note that these code examples are partial and somewhat simplified. Also, I do realize that my use of Objects as Associative Arrays is not best practice)

The result of this modification was MUCH closer, because the videos were now loaded by the time they were required to play, but there was still a short and inconsistent gap between the videos.

Attempt #3 I replaced the 'ended' event listener with a 'timeupdate' listener, beginning as follows:

nextVideo.addEventListener("timeupdate", function (event)
{
    if (this.currentTime > (this.duration - 0.1)  && this.currentTime > 1)
    {
        this.removeEventListener("timeupdate", function () { return; }, false);

('this.currentTime > 1' is simply to ensure that the previous video actually plays)

My hope was that the gap between videos was close enough to being consistent that starting the next video an arbitrary 0.1 seconds before the previous video ended would correct the gap to an acceptable extent. The result was that the timing was indeed better, but it was skipping videos. I attribute the skipped videos to misfiring of the 'timeupdate' event, but I could be mistaken.

Other alternative options I had also considered: SMIL Script (seems to be obsolete, and would likely have the same syncing issues anyway) ffmpeg on the backend to concatenate the videos (my host will not allow shell scripts)

FYI I am developing for Safari 5.0.3 at the moment

Answer 1

I had a similar problem that I managed to solve now thanks to your hints. The result is a possibly dynamic list of video elements that may be played back one after another without gaps.

Instead of a native Video Tag I use jwplayer on several video elements. On startup all elements begin to play one second, are paused and rewound to zero. Then one by another is played and made visible with display: block.

<ul class="playlist">
    <li>
        <video class="playlist" id="vid01" poster="img/preview.jpg">
            <source src="vid/v01.mp4" type="video/mp4">
        </video>
    </li>
    <li>
        <video class="playlist" id="vid02" poster="img/preview.jpg">
            <source src="vid/v02.mp4" type="video/mp4">
        </video>
    </li>
    <li>
        <video class="playlist" id="vid03" poster="img/preview.jpg">
            <source src="vid/v03.mp4" type="video/mp4">
        </video>
    </li>
    <li>
        <video class="playlist" id="vid04" poster="img/preview.jpg">
            <source src="vid/v04.mp4" type="video/mp4">
        </video>
    </li>
</ul>

On added to stage, 'preload' the video one or two seconds: (Excerpt)

var isPrebuffering = false,
    isSeeking = false;

$player = jwplayer(videoId).setup({
    width: '800px',
    height: '450px',
    autoplay: false,
    modes: [
        { type: 'html5' }
    ],
    events: {
        onReady: function(e) {
            prebuffer();
        },
// There is a quirk in jwplayer that makes it impossible to pause at the 
// new position directly after seeking. So we have to work around that.
        onTime: function(e) {
            if((e.position > 1.0 && isPrebuffering) || isSeeking) {
                finishPrebuffering();
            }
        }
    }
});

function prebuffer() {
    isPrebuffering = true;
    $player.setMute(true);
    $player.play();
};

function finishPrebuffering() {
    if(isPrebuffering) {
        isPrebuffering = false;
        isSeeking = true;
        $player.seek(0);
    }

    if($player.getPosition() === 0) {
        isSeeking = false;
        $player.pause();
        $player.setMute(false);
    }
};