HTML 5 currentTime accuracy

Question

I'm working on a project where we are using the value from the video element's currentTime property to perform processing on the server backend using ffmpeg. I've come across an issue where the video element seems to report a time code that is slightly different from the time code ffmpeg needs to access the correct point in the video.

So for instance in Firefox if the currentTime property reports that the current video time is 26.83 I might find that the frame I really want ended at 26.72 and so if I use the time to extract a frame using ffmpeg on the server I get the next frame instead of the current frame.

The amount of offset seems to be slightly different in different parts of the video and in different videos. But the offset is usually close to one tenth of a second in Firefox. In chrome the currentTime actually seems to be ahead or the correct currentTime by about 5 hundredths of a second. It's more difficult to figure out the offset in IE because the place where the frame shifts seems to change as I enter different time codes to look for the exact time code where the frame changes.

I'm pretty sure the time as used by ffmpeg is the correct time. It seems to agree more closely with other video editing software such as adobe premier.

Any ideas on what could be causing this behavior?

JS to get currentTime:

AVideo.prototype.getCurrentTime = function()
{
   return this.videoElement[0].currentTime;
};

Resulting ffmpeg command:

ffmpeg -y -i '/tmp/myVideo.mov' -vframes 1 -ss 2.4871 -f image2   -y '/tmp/myFrame.jpg' 2>&1

Answer 1

You are at the mercy of each browsers or players implementation here, and you are also in quite a tricky domain as timing in videos is complicated and you will likely see multiple different times being discussed.

Some examples are:

• DTS - decode time stamp, the time when a particular frame should be decoded
• PTS - presentation time stamp, the time when a particular frame should be presented. This is mostly likely the one you are interested in.
• SMPTE - traditional time stamp used in media. Good example of a link to online video here: http://allensarkisyan.github.io/VideoFrameDocs/

Taking a higher level view, the video will generally have a set 'frames per second' rate although this can sometimes be variable to further confuse things. Even when it is constant it is generally hard to calculate it in advance so it is an advantage if you actually know it (from metadata for the video for example).

Assuming constant fps, then the first thing that is useful to understand is that for a given frame rate, say 25 fps, you do not have enough frames to give a time stamp accurate to two decimal places of a second.

In other words, assuming a video starts at 00:00:00.00 then the first frame would in theory be at 00:00:00.00, the second at 00:00:00.00 + 1/25th of a second -> 00:00:00.04, the next frame at 00:00:00.08 etc.

So immediately we can see that the player has an issue if you ask it to seek to position 00:00:00.06 as it has to choose between the two closest frames, 00:00:00.04 and 00:00:00.08. In the worst case, for example when a scene in the movie is changing, these could be quite different looking frames.

It may be that a 'Frame accurate' video player would be more useful to you - i.e. if you pause the video at a certain place, then knowing the exact frame number of that place might allow you better align when you do your ffmpeg manipulation.

There is a good, if slightly old, BBC blog post about this here: http://www.bbc.co.uk/blogs/bbcinternet/2011/02/frame_accurate_video_in_html5.html and the SMPTE video link above may give you something to work with.

Some other useful reference for frame accurate players:

• https://stackoverflow.com/a/24829337/334402