how to capture html5 video programmatically (headless)

Question

I am trying to capture html5 video with js/css effects on it programmatically. I tried a couple of methods;

1. First I find this great blog post and implemented it. Everything was perfect till I found that phantomjs is not supporting html5 video tag so can not capture the video.

2. Second option was to use headless chrome to take continuous screenshots and feed these screenshots into ffmpeg to create the video. Although it worked to some level headless chromes screenshots was taking some time.. I couldn't create a smooth video..

3. On my third try I gave a chance to chrome's Page.startScreencast api. It could get video capture but frame rates was really problematic. The reason is that..

4. Now I am working on xvfb + chrome/firefox + ffmpeg combination for capturing video as mentioned on that comment. Theoretically it is promising but I couldn't managed to capture a video. Instead I have black screen..

My setup is below:

• light-http server having a simple video (web) within html5 video tag; on localhost
• start xvfb with Firefox and navigate to localhost/index.html ( video is there ) xvfb-run --listen-tcp --server-num 44 --auth-file /tmp/xvfb.auth -s "-ac -screen 0 1440x685x24" firefox --headless http://localhost

• start ffmpeg with x11grab parameter to grab frames from xvfb ffmpeg -f x11grab -video_size 1440x685 -i :44 -codec:v libx264 -r 12 ./output.mp4

• the result is black video :)

what should be the problem, how can I debug the problem?

---

ps: there is one more possible solution which I didn't tried yet. As phantomjs has capability to capture canvas; it may be possible to

• put video into canvas
• then capture the canvas

It seems like a dirty workaround that is why not tried yet..

---

UPDATE-1

Tried to get screenshot with xwd -root -silent -display :44 -out screen.xwdand than convert to jpeg convert screen.xwd shot.jpg the result is black jpg..

Answer 1

If you don't need the "live video output" I would suggest capturing video frame by frame. Here is the speech about some existing application https://vimeo.com/342829825

Otherwise, if life stream is required, you may go with the way that obs-linuxbrowser is doing. So cef + ffmpeg.