How to generate video thumbnails and preview them while hovering on the progress bar?

Question

I want to generate video thumbnail and preview it while hovering on the progress bar like YouTube video:

I've tried to test with videojs-thumbnails but failed. The README file doesn't contain enough information to fix it.

I've also tried to search on Google with keyword: video thumbnail progress bar. There are some related questions on SO but I can't find the solution for this case.

I found a javascript library videojs which contains event hovering on progress bar:

videojs('video').ready(function () {
    $(document).on('mousemove', '.vjs-progress-control', function() { 
        // How can I generate video thumbnails and preview them here?
    });
});

Answer 1

I struggled with this exact issue, about a year ago. Based on the thread here: How to generate video preview thumbnails for use in VideoJS? I finally concluded to go with offline generation of thumbnails, as it's much easier than trying to extract them on-the-fly.

I did a technical discussion/explanation of that struggle here: http://weasel.firmfriends.us/GeeksHomePages/subj-video-and-audio.html#implementing-video-thumbnails

My prototype example is here: https://weasel.firmfriends.us/Private3-BB/

EDIT:Also, I couldn't solve how to bind to the existing seekBar in video-js, so I added my own dedicated slider to view the thumbnails. That decision was mostly based on the need to use 'hover'/'onMouseOver' if one wants to use video-js's seekbar, and those gestures don't translate well to touch-screens (mobile devices).

EDIT: I've now solved the issue of how to bind the existing seekBar, so I've added that logic to my prototype example mentioned above.

Cheers. Hope this helps.

Answer 2

Currently (Dec. 2019), there are not so much javascript (both free version and paid version) library which supports adding thumbnail while hovering on video progress bar.

But you can follow on the road of videojs. They've already supported adding tooltip while hovering on video progress bar. Everything else you can do is Generating video thumbnails and adding them into the control bar for previewing.

In this example, we will explain about how to generate video thumbnail from <input type="file" />. Athough we can use video source with a directly link, in testing period, we have some problem with Tainted canvases may not be exported because of using canvas.toDataURL()

After videojs completes initializing, you can clone a new video from the source and append it to the body. Just to play and catch loadeddata event:

videojs('video').ready(function () {
    var that = this;

    var videoSource = this.player_.children_[0];

    var video = $(videoSource).clone().css('display', 'none').appendTo('body')[0];

    video.addEventListener('loadeddata', async function() { 
        // asynchronous code...
    });

    video.play();
});

Like YouTube video thumbnail, we will generate thumbnail file as an image. This image has a size:

(horizontalItemCount*thumbnailWidth)x(verticalItemCount*thumbnailHeight) = (5*158)x(5*90)

So 790x450 is the size of the image which contains 25 sub-thumbnails (YouTube uses 158 as the width and 90 as the height of the thumbnail). Like this:

Then, we will take video snapshot based on video duration. In this example, we generate thumbnail per second (each second has a thumbnail).

Because generating video thumbnail needs a long time based on video duration and quality, so we can make a default thumbnail with a dark theme for waiting.

.vjs-control-bar .vjs-thumbnail {
  position: absolute;
  width: 158px;
  height: 90px;
  top: -100px;
  background-color: #000;
  display: none;
}

After getting video duration:

var duration = parseInt(that.duration());

we need to parse it to an int before using in the loop because the value may be 14.036.

Everything else is: Setting the currentTime value of the new video and converting the video to canvas.

Because 1 canvas element can contains maximum 25 thumbnails by default, we have to add 25 thumbnails to the canvas one-by-one (from left to right, from top to bottom). Then we store it in an array.

If there is still another thumbnail, we create another canvas and repeat the action

var thumbnails = [];

var thumbnailWidth = 158;
var thumbnailHeight = 90;
var horizontalItemCount = 5;
var verticalItemCount = 5;

var init = function () {
    videojs('video').ready(function() {
        var that = this;

        var videoSource = this.player_.children_[0];

        var video = $(videoSource).clone().css('display', 'none').appendTo('body')[0];

        // videojs element
        var root = $(videoSource).closest('.video-js');

        // control bar element
        var controlBar = root.find('.vjs-control-bar');

        // thumbnail element
        controlBar.append('<div class="vjs-thumbnail"></div>');

        //
        controlBar.on('mousemove', '.vjs-progress-control', function() {
            // getting time 
            var time = $(this).find('.vjs-mouse-display .vjs-time-tooltip').text();

            // 
            var temp = null;

            // format: 09
            if (/^\d+$/.test(time)) {
                // re-format to: 0:0:09
                time = '0:0:' + time;
            } 
            // format: 1:09
            else if (/^\d+:\d+$/.test(time)) {
                // re-format to: 0:1:09
                time = '0:' + time;
            }

            //
            temp = time.split(':');

            // calculating to get seconds
            time = (+temp[0]) * 60 * 60 + (+temp[1]) * 60 + (+temp[2]);

            //
            for (var item of thumbnails) {
                //
                var data = item.sec.find(x => x.index === time);

                // thumbnail found
                if (data) {
                    // getting mouse position based on "vjs-mouse-display" element
                    var position = controlBar.find('.vjs-mouse-display').position();

                    // updating thumbnail css
                    controlBar.find('.vjs-thumbnail').css({
                        'background-image': 'url(' + item.data + ')',
                        'background-position-x': data.backgroundPositionX,
                        'background-position-y': data.backgroundPositionY,
                        'left': position.left + 10,
                        'display': 'block'
                    });

                    // exit
                    return;
                }
            }
        });

        // mouse leaving the control bar
        controlBar.on('mouseout', '.vjs-progress-control', function() {
            // hidding thumbnail
            controlBar.find('.vjs-thumbnail').css('display', 'none');
        });

        video.addEventListener('loadeddata', async function() {            
            //
            video.pause();

            //
            var count = 1;

            //
            var id = 1;

            //
            var x = 0, y = 0;

            //
            var array = [];

            //
            var duration = parseInt(that.duration());

            //
            for (var i = 1; i <= duration; i++) {
                array.push(i);
            }

            //
            var canvas;

            //
            var i, j;

            for (i = 0, j = array.length; i < j; i += horizontalItemCount) {
                //
                for (var startIndex of array.slice(i, i + horizontalItemCount)) {
                    //
                    var backgroundPositionX = x * thumbnailWidth;

                    //
                    var backgroundPositionY = y * thumbnailHeight;

                    //
                    var item = thumbnails.find(x => x.id === id);

                    if (!item) {
                        // 

                        //
                        canvas = document.createElement('canvas');

                        //
                        canvas.width = thumbnailWidth * horizontalItemCount;
                        canvas.height = thumbnailHeight * verticalItemCount;

                        //
                        thumbnails.push({
                            id: id,
                            canvas: canvas,
                            sec: [{
                                index: startIndex,
                                backgroundPositionX: -backgroundPositionX,
                                backgroundPositionY: -backgroundPositionY
                            }]
                        });
                    } else {
                        //

                        //
                        canvas = item.canvas;

                        //
                        item.sec.push({
                            index: startIndex,
                            backgroundPositionX: -backgroundPositionX,
                            backgroundPositionY: -backgroundPositionY
                        });
                    }

                    //
                    var context = canvas.getContext('2d');

                    //
                    video.currentTime = startIndex;

                    //
                    await new Promise(function(resolve) {
                        var event = function() {
                            //
                            context.drawImage(video, backgroundPositionX, backgroundPositionY, 
                                thumbnailWidth, thumbnailHeight);

                            //
                            x++;

                            // removing duplicate events
                            video.removeEventListener('canplay', event);

                            // 
                            resolve();
                        };

                        // 
                        video.addEventListener('canplay', event);
                    });


                    // 1 thumbnail is generated completely
                    count++;
                }

                // reset x coordinate
                x = 0;

                // increase y coordinate
                y++;

                // checking for overflow
                if (count > horizontalItemCount * verticalItemCount) {
                    //
                    count = 1;

                    //
                    x = 0;

                    //
                    y = 0;

                    //
                    id++;
                }

            }

            // looping through thumbnail list to update thumbnail
            thumbnails.forEach(function(item) {
                // converting canvas to blob to get short url
                item.canvas.toBlob(blob => item.data = URL.createObjectURL(blob), 'image/jpeg');

                // deleting unused property
                delete item.canvas;
            });

            
            
            console.log('done...');
        });

        // playing video to hit "loadeddata" event
        video.play();
    });
};

$('[type=file]').on('change', function() {
    var file = this.files[0];
    $('video source').prop('src', URL.createObjectURL(file));

    init();
});

.vjs-control-bar .vjs-thumbnail {
  position: absolute;
  width: 158px;
  height: 90px;
  top: -100px;
  background-color: #000;
  display: none;
}

<link rel="stylesheet" href="https://vjs.zencdn.net/7.5.5/video-js.css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.2.4/jquery.min.js"></script>
<script src="https://vjs.zencdn.net/7.5.5/video.js"></script>

<input type="file" accept=".mp4" />
<video id="video" class="video-js vjs-default-skin" width="500" height="250" controls> 
    <source src="" type='video/mp4'>
</video>

Fiddle