Improved thumbnail extraction from videos

Question

I have been using FFmpeg to find the middle frame of a h264 video file, and extract the jpg thumbnail for use on a streaming portal. This is done automatically for each uploaded video.

Sometimes the frame happens to be a black frame or just semantically bad i.e. a background or blurry shot which doesn't relate well to the video content.

I wonder if I can use openCV or some other method/library to programmatically find better thumbnails through facial recognition or frame analysis.

Answer 1

I've run into that problem myself and came up with a very crude-yet-simple algorithm to ensure my thumbnails were more "interesting". How?

1. Create x-number of thumbnails all at different points. E.g. 5 thumbnails
2. Use the largest (in bytes) file and discard the rest

Why does this work? Because jpeg files of a monotone 'boring' image, like an all black screen, compress into a much smaller files than an image with many objects and colors in it.

It's not perfect, but is a viable 80/20 solution. (Solves 80% of the problem with 20% of the work.) Coding something that actually analyzes the image itself is going to be considerably more work.

Answer 2

Libavfilter has got a thumbnail filter, which is meant to pick the most representative frame from a series of frames. Not sure how it works, but heres the docs http://ffmpeg.org/libavfilter.html#thumbnail