Video Seek Scroll (60FPS)

Question

Trying to achieve an effect of seeking through a video when the page is scrolled. This has been achieved by exporting all frames of the video to JPEG images, pre-loading the images, and rendering them to a canvas. However, this approach uses a lot of bandwidth and takes a long time to load on slow networks.

Trying to achieve the same effect by rendering a video to a canvas does not play as smoothly as the image-based approach.

Here is a working demo with the video-based approach:

https://codesandbox.io/s/infallible-chaum-grvi0r?file=/index.html

Since HTMLMediaElement.fastSeek() doesn't have widespread browser coverage, how can one achieve a realtime playback rate of 30-60 FPS?

Here is the relevant code for the effect (see CSB link above for the full code):

const video = document.querySelector("video");
const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");

(function tick() {
  requestAnimationFrame(tick);

  const { scrollHeight, clientHeight, scrollTop } = document.body;
  const maxScroll = scrollHeight - clientHeight;
  const scrollProgress = scrollTop / maxScroll;

  canvas.width = document.body.clientWidth;
  canvas.height = document.body.clientHeight;

  // This is the line that causes the issue
  video.currentTime = video.duration * scrollProgress;

  ctx.clearRect(0, 0, canvas.width, canvas.height);
  ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
})();

Answer 1

The problem is due to specifics of drawImage() as noted here, it should be called only after firing of seeked event. The code below solves the problem by using a sentinel value (seeked = false) which will block the execution of drawImage() until the next seeked event:

const video = document.querySelector("video");
const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");
let seeked;
(function tick() {
    requestAnimationFrame(tick);
    if (seeked) {
        seeked = false;
        const { scrollHeight, clientHeight, scrollTop } = document.body;
        const maxScroll = scrollHeight - clientHeight;
        const scrollProgress = scrollTop / Math.max(maxScroll, 1);

        canvas.width = document.body.clientWidth;
        canvas.height = document.body.clientHeight;

        // This is the line that causes the issue
        video.currentTime = video.duration * scrollProgress;

        ctx.clearRect(0, 0, canvas.width, canvas.height);
        ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
    }
})();
video.addEventListener('seeked', () => {
    seeked = true;
});
video.currentTime = .001;