How to extract slides from a video using python

Question

I have a video training course supplied as AVI files. Most of the screens are shown as slides with a mouse pointer moving around on them.

I'd like to capture a screenshot of the slide automatically when the screen changes (ignoring when the image changes a small amount due to the mouse pointer moving around.)

I want to do this so I can paste the images into a word or html document that I can add notes to as I learn as at the moment I'm taking screenshots but it's very slow and tedious and the course is very long (around 24 hours total play time).

I know python well but am unsure as to how I would go about extracting stills from a video file and then how to compare one still with another to see how much they differ to decide which to keep and which to discard.

Can anyone suggest how to go about doing this?

Answer 1

A tool like ffmpeg is suited for extracting images from a video. From the manual:

 ffmpeg -i foo.avi -r 1 -s WxH -f image2 foo-%03d.jpeg

This will extract one video frame per second from the video and will output them in files named foo-001.jpeg, foo-002.jpeg, etc. Images will be rescaled to fit the new WxH values.

Comparing them for differences can then perhaps be done by PIL and/or OpenCV.

EDIT: I just realized that it probably would be more efficient to only grab the key frames (intra frame), because those occur when a drastic change in the scene happens. A quick google later we have this:

ffmpeg -i foo.avi -vsync 0 -vf select="eq(pict_type\,PICT_TYPE_I)" -s WxH -f image2 foo-%03d.jpeg

Answer 2

What you basically want is scene detection. framedifferenceanalyzer is an educational proof of concept in Python that does exactly that, and should provide a good starting point for learning about the problem itself.

As for implementing it yourself, ffmpeg is the ideal tool for converting a video into a sequence of frames - I probably wouldn't attempt doing that part in pure Python.

For calculating the difference between frames you could maybe use ImageMagick (its compare tool in particular). There are several Python bindings for ImageMagick, for example PythonMagick or magickwand to name just two.

You could also use OpenCV to do the image analysis. OpenCV is a library of high performance, high quality computer vision algorithms and probably one of, if not the most powerful tool out there to do things like this. However, it kind of assumes that you have a certain knowledge about computer vision / image processing and already have a good idea of what you're looking for.