OpenCV doesn't report accurate frame rate/count

Question

I have a 33 second video that I'm trying to process with OpenCV. My goal is to determine what instance in time (relative to the start of the video) each frame corresponds to. I'm doing this in order to be able to compare frames from videos of the same scene that have been recorded at different frame rates.

What's working:

• The FPS is correctly reported as 59.75. This is consistent with what ffprobe reports, so I'm happy to believe that's correct.

The problems I'm having are:

• CAP_PROP_POS_MSEC returns incorrect values. By the end of the video, it's up to 557924ms (over 9 min). For a 33s video, this can't be right.
• CAP_PROP_FRAME_COUNT is also incorrect. It's reported as 33371, which at 59.75 fps would give over 9 minutes worth of footage. Consistent with the above error, but still incorrect.
• CAP_PROP_POS_FRAME is similarly incorrect.

The video can be found here (approx. 10MB).

Any ideas on what could be wrong?

ffprobe output:

FFprobe version SVN-r20090707, Copyright (c) 2007-2009 Stefano Sabatini
  libavutil     49.15. 0 / 49.15. 0
  libavcodec    52.20. 0 / 52.20. 1
  libavformat   52.31. 0 / 52.31. 0
  built on Jan 20 2010 00:13:01, gcc: 4.4.3 20100116 (prerelease)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/misha/Dropbox/Public/sequence.mp4':
  Duration: 00:00:33.37, start: 0.000000, bitrate: 2760 kb/s
    Stream #0.0(und): Video: h264, yuv420p, 1920x1080, 59.75 tbr, 1k tbn, 2k tbc
    Stream #0.1(und): Audio: aac, 44100 Hz, stereo, s16

Full code:

#include <iostream>
#include <assert.h>

#include <cv.h>    
#include <highgui.h>

#include <cmath>
#include <iostream>
#include <string.h>
#include <stdio.h>

extern "C"
{
#include "options.h"
}

using namespace std;

#define DEBUG 0

static void
print_usage(char *argv0)
{
    cerr << "usage: " << argv0 << " video.avi [options]" << endl;
}

int main(int argc, char** argv)
{
    if (argc < 2)
    {
        print_usage(argv[0]);
        return -1;
    }

    int           step      = 30;
    struct Option options[] =
    {
        { "step",        1, &step },
        { NULL,          0, NULL }
    };

    int ret = parse_options(2, argc, argv, options);
    if (ret == 0)
    {
        print_usage(argv[0]);
        return -1;
    }

    CvCapture *capture = cvCaptureFromFile(argv[1]);
    int counter = 0;
    while (cvGrabFrame(capture))
    {
        ++counter;

        IplImage *frame = cvRetrieveFrame(capture);
        double millis   = cvGetCaptureProperty(capture, CV_CAP_PROP_POS_MSEC);
        double current  = cvGetCaptureProperty(capture, CV_CAP_PROP_POS_FRAMES);
        double total    = cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_COUNT);
        printf("%d %d/%d %f\n", counter, (int)current, (int)total, millis);
        int min = (int)(millis/1000/60);
        millis -= min*60000;
        int sec = (int)(millis/1000);
        millis -= sec*1000;

        printf("%d %02d:%02d:%f\n", counter, min, sec, millis);
    }
    cvReleaseCapture(&capture);

    return 0;
}

Answer 1

Always inform the software version you are using: which OpenCV version are you using for that? Yours might be an old version, so update to the most recent if possible.

If your video file is part of some other larger video, this information might actually be correct, since:

CV_CAP_PROP_POS_MSEC - film current position in milliseconds or video capture timestamp

OpenCV might be simply reading all these stuff from the file header, which is obviously wrong. This could happen if someone used an ax (or other medieval tool) to strip this piece from the original video.

You should try with videos that you made and you know they haven't been tampered with.

Worst case scenario, you will have to implement these features yourself. No biggie.

EDIT: @misha I didn't notice at first that you are using:

CvCapture *capture = cvCaptureFromFile(argv[1]);

Replace it for cvCaptureFromAVI() if you can, and ALWAYS check the return value of OpenCV calls:

CvCapture *capture = cvCaptureFromAVI(argv[1]);
if(!capture) 
{
    printf("!!! cvCaptureFromAVI failed (file not found?)\n");
    return -1; 
}

A few days ago I shared a code that uses OpenCV to read a video file and then save the frames as JPG images on the disk. It also reports the current FPS using the traditional cvGetCaptureProperty(capture, CV_CAP_PROP_FPS); so it could be interesting for you to take a look at that. Go check it out.

EDIT:

You should also check this thread about frame count using OpenCV;

By the way, I just updated some libraries on my Ubuntu system and recompiled OpenCV-2.1.0.tar.bz2 (using cmake). I changed my source code (that uses cvCaptureFromAVI()) to print stuff using your debug method on each frame. It seems it works:

* Filename: sequence.mp4
* FPS: 59
...
...
...
17 00:00:567.000000
18 601/33371 601.000000
18 00:00:601.000000
19 634/33371 634.000000
19 00:00:634.000000
20 668/33371 668.000000
20 00:00:668.000000
21 701/33371 701.000000
21 00:00:701.000000
22 734/33371 734.000000
22 00:00:734.000000
23 768/33371 768.000000
23 00:00:768.000000
24 801/33371 801.000000
24 00:00:801.000000
25 835/33371 835.000000
25 00:00:835.000000
26 868/33371 868.000000
26 00:00:868.000000
27 901/33371 901.000000
27 00:00:901.000000
28 935/33371 935.000000
...