Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reading video frame-by-frame under iOS

I'm looking for a way to retrieve the individual frames of a video using iOS API. I tried using AVAssetImageGenerator but it seems to only provide frame to the nearest second which is a bit too rough for my usage.

From what I understand of the documentation, a pipeline of AVAssetReader, AVAssetReaderOutput and CMSampleBufferGetImageBuffer I should be able to do something but I'm stuck with a CVImageBufferRef. With this I'm looking for a way to get a CGImageRef or a UIImage but haven't found it.

Real-time is not needed and the more I can stick to provided API the better.

Thanks a lot!

Edit: Based on this site: http://www.7twenty7.com/blog/2010/11/video-processing-with-av-foundation and this question: how to convert a CVImageBufferRef to UIImage I'm nearing on a solution. Problem, the AVAssetReader stops reading after the first copyNextSampleBuffer without giving me anything (the sampleBuffer is NULL).

The video is readable by MPMoviePlayerController. I don't understand what's wrong.

like image 879
hlidotbe Avatar asked Jul 21 '11 21:07

hlidotbe


People also ask

How do I view a video frame by frame on my iPhone?

If you are having trouble editing to an exact frame in the video, you can edit frame by frame by pressing and holding a finger on either end of the yellow bar. You will see the video spread out, revealing individual frames. Slide the end of the bar to either include or cut out video frame(s).

How do I play a video frame by frame on a Mac?

Open the video file you want to watch, and press E on your keyboard. This will pause the video. Now, everytime you hit 'E', the video will advance frame by frame.

How do I advance video frame by frame?

YouTube frame advance (forward): Pause the video and press “.” to go to the next frame. YouTube frame advance (backward): Pause the video and press “,” to go to the last frame.


2 Answers

The two links above actually answer my question and the empty copyNextBufferSample is an issue with iOS SDK 5.0b3, it works on the device.

like image 200
hlidotbe Avatar answered Oct 21 '22 10:10

hlidotbe


AVAssetImageGenerator has very loose default tolerances for the exact frame time that is grabbed. It has two properties that determine the tolerance: requestedTimeToleranceBefore and requestedTimeToleranceAfter. These tolerances default to kCMTimePositiveInfinity, so if you want exact times, set them to kCMTimeZero to get exact frames.

(It may take longer to grab the exact frames than approximate frames, but you state that realtime is not an issue.)

like image 31
brainjam Avatar answered Oct 21 '22 10:10

brainjam