Python scene change detection

Question

I am wondering if anyone has any experience with Python and video processing. Essentially, I would like to know if there are any libraries that would allow me to do scene detection in a video? If not, are there any that can allow me to split the video up into a series of frames and let me mess about with the pixels?

Thanks!

Answer 1

OpenCV has Python bindings; I don't think it has any scene boundary algorithms / functions built it, but you can definitely use it to write your own.

Answer 2

You can use FFmpeg to do the scene detection and obtain the change frames and their timestamps. The command can be combined with a python script and you can modify it according to your use case.

You can simply use the command:

ffmpeg inputvideo.mp4 -filter_complex "select='gt(scene,0.3)',metadata=print:file=time.txt" -vsync vfr img%03d.png

This will save just the relevant information in the time.txt file like below and also save the shot change images in order:

frame:0    pts:108859  pts_time:1.20954
lavfi.scene_score=0.436456
frame:1    pts:285285  pts_time:3.16983
lavfi.scene_score=0.444537
frame:2    pts:487987  pts_time:5.42208
lavfi.scene_score=0.494256
frame:3    pts:904654  pts_time:10.0517
lavfi.scene_score=0.462327
frame:4    pts:2533781 pts_time:28.1531
lavfi.scene_score=0.460413
frame:5    pts:2668916 pts_time:29.6546
lavfi.scene_score=0.432326

The frame is the serial number of the detected shot change from the starting. Also, choose your threshold value (here 0.3) appropriately for your use case to get correct outputs