How to record an HTML animation and save it as a video, in an automated manner in the backend

Question

I need to record a webpage and save it as a video, in an automated manner, without human interaction.

I am creating a NodeJS app that generates MP4 videos on the request of the user. The user provides an MP3 file, the app generates animated waveforms for the sound file on top of an illustration.

What I came up with so far is a system that opens a generated web page in the backend, plays the audio file, and shows audio visualization for the audio file on an HTML canvas element. On top of another canvas with mainly static components, such as images, that do not animate. The system records this, the output will be a video file. Finally, I will merge the video file with the sound file to create the final file for the user.

I came up with 2 possible solutions but both of them have problems which I am not able to solve at the moment.

---

Solution #1

Use a headless browser API such as Phantomjs or Puppeteer to snatch a screenshot x time every second and pipe it to FFmpeg.

The problem

The problem with this is that the process is not realtime. It would work fine if it's JUST an animation but mine is dependant on the audio file. The audio file will play-on during the render which results in a glitchy 1FPS-esque video.

Possible solution?

Don't play the audio file live but convert the audio file into raw data. Animate the audio visualization based on the raw data instead. Not sure how to do this and if it's even possible.

---

Solution #2

Play, record, and save the animation, all in the frontend. Could use ccapture.js to record and save a canvas. Use a headless browser to open the page and save it to disk when it's done playing. Doesn't sound like it's the best solution.

The problem(s)

I have more than 1 canvas. It takes a while, especially when the audio file is longer than 10 minutes. Making users wait for a long time can be a deal-breaker.

Possible solution?

Merge canvases into one.

No idea how to speed up the rendering time and I doubt it's possible this way.

Answer 1

Late answer from someone looking for similar options due to the convenience of some browser SVG APIs:

My first recommendation, as someone who has written a fair amount of my own audio visualization software, is to use a graphics library and language that don't require a browser or GPU, like Gd or Anti-grain Geometry or Cairo with any server-side language. You might also check out Processing.org (which I haven't used), not sure if there's a headless version.

If that's not possible, I've found these so far but haven't tried them:

• https://github.com/tungs/timecut

• https://github.com/myplanet/headless-render

• https://wave.video/blog/how-we-render-animated-content-from-html5-canvas/