Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

OpenCV tracking using optical flow

I use this to functions as a base of my tracking algorithm.

    //1. detect the features
    cv::goodFeaturesToTrack(gray_prev, // the image 
    features,   // the output detected features
    max_count,  // the maximum number of features 
    qlevel,     // quality level
    minDist);   // min distance between two features

    // 2. track features
    cv::calcOpticalFlowPyrLK(
    gray_prev, gray, // 2 consecutive images
    points_prev, // input point positions in first im
    points_cur, // output point positions in the 2nd
    status,    // tracking success
    err);      // tracking error

cv::calcOpticalFlowPyrLK takes vector of points from the previous image as input, and returns appropriate points on the next image. Suppose I have random pixel (x, y) on the previous image, how can I calculate position of this pixel on the next image using OpenCV optical flow function?

like image 406
Alex Hoppus Avatar asked Mar 14 '12 11:03

Alex Hoppus


People also ask

What is optical flow tracking?

Optical flow is one of the efficient approaches to track the movement of objects. Optical flow studies the relative motion of objects across different frame sequences based on the velocity of movement of objects and illumination changes.

What is optical flow in Opencv?

Optical flow is the pattern of apparent motion of image objects between two consecutive frames caused by the movement of object or camera. It is 2D vector field where each vector is a displacement vector showing the movement of points from first frame to second.

How does optical flow work?

Optical flow is a technique used to describe image motion. It is usually applied to a series of images that have a small time step between them, for example, video frames. Optical flow calculates a velocity for points within the images, and provides an estimation of where points could be in the next image sequence.

What is optical flow in Python?

Optical flow is the motion of objects between the consecutive frames of the sequence, caused by the relative motion between the camera and the object. It can be of two types-Sparse Optical flow and Dense Optical flow.


2 Answers

As you write, cv::goodFeaturesToTrack takes an image as input and produces a vector of points which it deems "good to track". These are chosen based on their ability to stand out from their surroundings, and are based on Harris corners in the image. A tracker would normally be initialised by passing the first image to goodFeaturesToTrack and obtaining a set of features to track. These features could then be passed to cv::calcOpticalFlowPyrLK as the previous points, along with the next image in the sequence and it will produce the next points as output, which then become input points in the next iteration.

If you want to try to track a different set of pixels (rather than features generated by cv::goodFeaturesToTrack or a similar function), then simply provide these to cv::calcOpticalFlowPyrLK along with the next image.

Very simply, in code:

// Obtain first image and set up two feature vectors
cv::Mat image_prev, image_next;
std::vector<cv::Point> features_prev, features_next;

image_next = getImage();

// Obtain initial set of features
cv::goodFeaturesToTrack(image_next, // the image 
  features_next,   // the output detected features
  max_count,  // the maximum number of features 
  qlevel,     // quality level
  minDist     // min distance between two features
);

// Tracker is initialised and initial features are stored in features_next
// Now iterate through rest of images
for(;;)
{
    image_prev = image_next.clone();
    feature_prev = features_next;
    image_next = getImage();  // Get next image

    // Find position of feature in new image
    cv::calcOpticalFlowPyrLK(
      image_prev, image_next, // 2 consecutive images
      points_prev, // input point positions in first im
      points_next, // output point positions in the 2nd
      status,    // tracking success
      err      // tracking error
    );

    if ( stopTracking() ) break;
}
like image 135
Chris Avatar answered Sep 17 '22 15:09

Chris


cv::calcOpticalFlowPyrLK(..) function uses arguments :

cv::calcOpticalFlowPyrLK(prev_gray, curr_gray, features_prev, features_next, status, err);

cv::Mat prev_gray, curr_gray;
std::vector<cv::Point2f> features_prev, features_next;
std::vector<uchar> status;
std::vector<float> err;

simplest(partial) code to find pixel in next frame :

features_prev.push_back(cv::Point(4, 5));
cv::calcOpticalFlowPyrLK(prev_gray, curr_gray, features_prev, features_next, status, err);

If pixel was successfully found status[0] == 1 and features_next[0] will show coordinates of pixel in next frame. Value information can be found in this example: OpenCV/samples/cpp/lkdemo.cpp

like image 37
MSeskas Avatar answered Sep 19 '22 15:09

MSeskas