Number of cuts in a 30min video file

Question

I have a video that is 30min long (encoded mp4:h264) and I need to count the numbers of cuts in the movie. It should include cuts as well as cross-fades.

So something quite similar to i-Frame detection....

I have available Linux with ffmpeg / libav as well as a Windows with Adobe Premiere.

Any clues? Or other software?

Answer 1

As other answers have pointed out, obtaining good accuracy with an automatic solution is difficult. Crossfades, zooms, pans, etc... all make it difficult for an automatic tool to conclude if consecutive frames belong to the same scene or not.

That being said, I would try to use the OpenCV library to analyze differences between consecutive frames and try to determine, using some empirical threshold, if they are similar enough to be considered from the same scene (choose the interval of frames, not necessarily every frame).

It's easy to extract frames from a video using the VideoCapture class.

Once frames are extracted you can use various methods to try to decide if two frames are related.

A few ideas:

1. Use the Lucas-Kanade method to find the optical flow between frames and conclude if the difference indeed results from motion of objects in the frames, or if it's completely different scenes.
2. Use a features detection algorithm (SIFT, SURF, etc) to characterize the frame and compare it to other frames. See also discussion here.

Good luck!

Answer 2

This is a classic computer vision problem. The search term is 'video scene segmentation', 'video scene extraction', 'automated video scene segmentation' or 'shot boundary detection'. You should check the literature on that because there is no silver bullet solution. As many techniques applied in computer vision, segmentation highly depends on the content of the image/video and the features which can be extracted from it.

That said expressive features (e.g. extreme change in hue or brightness) can be easier detected than non expressive features (soccer game with a slight change). You will likely find some simple MATLAB code which can be converted into OpenCV code.

There are annotated datasets to test the yout algorithms.

Also take a look at: What is the best way to divide a video into scenes (segments)