How to read mp4 video to be processed by scikit-image?

Question

I would like to apply a scikit-image function (specifically the template matching function match_template) to the frames of a mp4 video, h264 encoding. It's important for my application to track the time of each frame, but I know the framerate so I can easily calculate from the frame number.

Please note that I'm running on low resources, and I would like to keep dependencies as slim as possible: numpy is needed anyway, and since I'm planning to use scikit-image, I would avoid importing (and compiling) openCV just to read the video.

I see at the bottom of this page that scikit-image can seamleassly process video stored as a numpy array, obtaining that would thus be ideal.

Answer 1

Imageio python package should do what you want. Here is a python snippet using this package:

import pylab import imageio filename = '/tmp/file.mp4' vid = imageio.get_reader(filename,  'ffmpeg') nums = [10, 287] for num in nums:     image = vid.get_data(num)     fig = pylab.figure()     fig.suptitle('image #{}'.format(num), fontsize=20)     pylab.imshow(image) pylab.show() 

You can also directly iterate over the images in the file (see the documentation ):

for i, im in enumerate(vid):     print('Mean of frame %i is %1.1f' % (i, im.mean())) 

To install imageio you can use pip:

pip install imageio 

An other solution would be to use moviepy (which use a similar code to read video), but I think imageio is lighter and does the job.

---

response to first comment

In order to check if the nominal frame rate is the same over the whole file, you can count the number of frame in the iterator:

count = 0 try:     for _ in vid:         count += 1 except RuntimeError:     print('something went wront in iterating, maybee wrong fps number') finally:     print('number of frames counted {}, number of frames in metada {}'.format(count, vid.get_meta_data()['nframes']))   In [10]: something went wront in iterating, maybee wrong fps number          number of frames counted 454, number of frames in metada 461 

In order to display the timestamp of each frame:

try:     for num, image in enumerate(vid.iter_data()):         if num % int(vid._meta['fps']):             continue         else:             fig = pylab.figure()             pylab.imshow(image)             timestamp = float(num)/ vid.get_meta_data()['fps']             print(timestamp)             fig.suptitle('image #{}, timestamp={}'.format(num, timestamp), fontsize=20)             pylab.show() except RuntimeError:     print('something went wrong')