Training Algorithm to train this data

Question

I am working in MATLAB

PLots

NOTE : Here, the data plotted is the track of x - position of the pixel at position (i,j) of the FIRST frame throughout all the frames. It means that the pixel at (23,87) in the first frame has, at the end of the sequence, x-position as 35 (as visible in the plot).

Here is some typical plots of x_pos for some different values of (i,j) . (i,j) refers to a pixel at (i,j) in the first frame not throughout all frames

For (i,j) = (23 ,87)

(i,j) = (42 ,56)

(i,j) = (67 ,19)

Answer 1

So it's not about pixels in the image, but more about moving object, which makes the task much more tractable. Your data is indeed time series, thus time-aware algorithms are preferable. Markov models (in particular Markov chains and a bit more sophisticated Hidden Markov models) are classic examples of them.

However, your input is noisy because of camera instability. Thus, even better solution would be to use Kalman filter - model similar to HMMs, but with explicit notion of noise. It is widely used in robotics, navigation and similar areas to estimate current and predict future position of a vehicle based on inexact sensor data and historical information. Doesn't it sound similar to what you need?

I'm not big fun of Matlab, but it seems to have kalman function that implements mentioned filter.

Answer 2

A video is like a sequence of photos of real objects.
And real object, in front of a camera, can do only 2 different things:

1. they stand still
2. they move

If the pixel you are trying to predict are from a video, then you need to look ad how pixel are moving on screen, because object are moving on screen.

And this is how video codec compression works (H264, H265...) (clearly video compression algorithm are much more complex that just try to understand the direction of a pixel... :-) )

Here is some question/answer on stackoverflow that may help you:

• Motion vectors calculation
• Kalman filter in computer vision: the choice of Q and R noise covariances
• How to do motion tracking of an object using video?
• Vehicle segmentation and tracking