Key Frame Extraction From Video

Question

I need Need To extract Key Frames from Video/Stream.So is there any standard implementation. I am using open CV. (Currently i am extracting frames each second which is slower i need to improve performance.) So if any one has optimized implementation please reply in here.

Answer 1

The ffmpeg solution should work well.

For someone facing problems with the select filter 'eq(pict_type\,PICT_TYPE_I)', you might want to try the filter as 'eq(pict_type\,I)'. This was broken for a while so some versions of ffmpeg might not recognize the constant. The fix can be seen here.

The final command which worked for me finally was:

ffmpeg -vf select='eq(pict_type\,I)' -i yourVideo.mp4 -vsync 2 -s 160x90 -f image2 thumbnails-%02d.jpeg

Answer 2

I will assume that a keyframe is a frame presenting content a lot different from the previous ones (it's not a formal definition, but it fits). Take frames i and i+1. Use cv2.absDiff to compute the difference between the frames and cv2.sumElems to get the sum of all pixels differences. Do this for all frames i. This will reduce your video stream to an one dimensional signal. Look for peaks in this signal and pick keyframes corresponding to these peaks. To find peaks choose a threshold on this signal either manually by finding a frame you deem to be key, and letting its error be the error threshold or automatically using statistics (e.g. any frame i+1 where the error is greater than 1 stdev from the mean error).