How to get a list of key frames (sync frames) time stamps for a video file in Android?

Question

Is there any API which can give me the list of time stamps of the key frames for the given video file in Android?

Answer 1

4.1+ - faster than mp4parser because done in native underneath and the code is cleaner

mp4parser you have to calculate the timestamps from the getsyncsamples + getsampledurations which is also annoying and they don't match up with ffprobe. this gets accurate results

MediaExtractor extractor = new MediaExtractor();
extractor.setDataSource(getVideoPath());

int trackindex = MediaExtractorUtil.selectVideoTrack(extractor);
extractor.selectTrack(trackindex);

while (extractor.getSampleTime() != -1) {
    long sampleTime = extractor.getSampleTime();

    // check not really necessary but JIC
    if ((extractor.getSampleFlags() & MediaExtractor.SAMPLE_FLAG_SYNC) > 0) {
        mKeyframeTimestampsMS.add(sampleTime / 1000);
    }

    extractor.seekTo(sampleTime + 1, MediaExtractor.SEEK_TO_NEXT_SYNC);
}

Answer 2

For an elegant solution, it seems as if you'd need to use ffmpeg for android to achieve this. Someone else tried it that way: How to get the frame from video file in android ; and was suggested to use ffmpeg.

I checked the reference manual, but could only find a way to get a keyframe itself (without timestamp) relative to a given position in time using the MediaMetadataRetriever() as explained here: http://developer.android.com/reference/android/media/MediaMetadataRetriever.html#extractMetadata%28int%29

If you don't want to use ffmpeg you could try the following:

private ArrayList<int> getKeyFrameTimestamps(String filename) {
  ArrayList<int> timeStamps = new ArrayList<int>();

  try {
    MediaMetadataRetriever mRetriever = new MediaMetadataRetriever();
    mRetriever.setDataSource(getDataSource(filename));
    mRetriever.getFrameAtTime(i, MediaMetadataRetriever.OPTION_CLOSEST_SYNC);

    Bitmap lastFame = mRetriever.getFrameAtTime(0, MediaMetadataRetriever.OPTION_NEXT_SYNC);
    result.add(0);

    for(int time=1; frame ; ++time) {
      Bitmap frame = mRetriever.getFrameAtTime(time, MediaMetadataRetriever.OPTION_PREVIOUS_SYNC);
      if (frame && !frame.sameAs(lastFrame)) {
        result.add(time);
        lastFrame = frame;
      }
    }
  } catch (Exception e) {  /* TODO: HANDLE THIS */ }

  return timeStamps;
}

This is of course not very elegant, but would probably work without hassle to install ffmpeg... It's probably also quite slow (I don't know). But maybe it's just what you need.

PS: An ffmpeg tutorial that fits your problem can be found here: http://dranger.com/ffmpeg/tutorial07.html

Answer 3

I believe the correct answer is http://code.google.com/p/mp4parser/

The isoparser API can read and write the MP4 file structure. It is a low level tool dealing with the so called boxes but it is as well as dealing with structure like tracks and movies.

Nice API and ways much faster than ffmpeg behemoth