record Linux Wayland/DRM screen using ffmpeg's kmsgrab device with superimposed webcam

Question

Setup is Linux, ffmpeg using kabylake iGPU.

I am capturing a running sway instance using the kmsgrab device, which requires the use of a hardware backend to coherently process the image on my hardware. Only VA API fits this bill. I want to overlay the webcam in the bottom right corner during encoding. However, attempts at manipulating the filter graph to accomplish this have been unsuccessful. This is ultimately for Twitch/Tube stream.

Right now, I am actually capturing the webcam to an sdl window, and simply recording the screen using separate instances of ffmpeg. This doesn't actually solve my problem since the window is easily disguised by workspace switching or other windows.

This is the workaround:

#!/usr/bin/env zsh

# record webcam and open it in sdl window
ffmpeg -v quiet -hide_banner \
  -re -video_size 640X480 -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i /dev/video0 \
  -vf 'format=nv12,hwupload' -c:v hevc_vaapi -f hevc - \
  | ffmpeg -v quiet -i - -f sdl2 - &

# wait for webcam window to open
until swaymsg -t get_tree | grep 'pipe:' &>/dev/null; do
  sleep 0.5
done

# position webcam in the bottom right corner of screen using sway
swaymsg floating enable
swaymsg resize set width 320 height 240
swaymsg move position 1580 795
swaymsg focus tiling

#screencast
ffmpeg -format bgra -framerate 60 -f kmsgrab -thread_queue_size 1024 -i - \
  -f alsa -ac 2 -thread_queue_size 1024 -i hw:0 \
  -vf 'hwmap=derive_device=vaapi,scale_vaapi=w=1920:h=1080:format=nv12' \
  -c:v h264_vaapi -g 120 -b:v 3M -maxrate 3M -pix_fmt vaapi_vld -c:a aac -ab 96k -threads $(nproc) \
  output.mkv

kill %1

Answer 1

in momento, this may be impossible.

It looks like a limitation in ffmpeg's vaapi backend. I noticed that Intel's QuickSync backend has an overlay_qsv filter, yet there is no equivalent overlay_vaapi.

In theory, there may be a mechanism to download the hardware surface to a software buffer using the hwdownload filter once it's been encoded in hevc_vaapi so as to make use of the software overlay filter to do the superimposition in the last phase of encoding. I will leave this open for now to see if someone can help work toward this end or knows anything more on the topic.

note on QuickSync:

After building with QuickSync support and attempting to use it, it appears to lack the capability to derive a surface from the kmsgrab device resulting in a function not implemented error from ffmpeg.