Calibration of images to obtain a top-view for points that lie on a same plane

Question

Calibration:

I have calibrated the camera using this vision toolbox in Matlab. I used checkerboard images to do so. After calibration I get the cameraParams which contains:

Camera Extrinsics
RotationMatrices: [3x3x18 double]
TranslationVectors: [18x3 double]

and

 Camera Intrinsics
 IntrinsicMatrix: [3x3 double]
 FocalLength: [1.0446e+03 1.0428e+03]
 PrincipalPoint: [604.1474 359.7477]
 Skew: 3.5436

Aim: I have recorded trajectories of some objects in motion using this camera. Each object corresponds to a single point in a frame. Now, I want to project the points such that I get a top-view.

1. Note all these points I wish to transform are are the on the same plane.

   ex: [xcor_i,ycor_i ]

   -101.7000  -77.4040
   -102.4200  -77.4040
   

2. KEYPOINT: This plane is perpendicular to one of images of checkerboard used for calibration. For that image(below), I know the height of origin of the checkerboard of from ground(193.040 cm). And the plane to project the points on is parallel to the ground and perpendicular to this image.

Code (Ref:https://stackoverflow.com/a/27260492/3646408 and answer by @Dima below):

function generate_homographic_matrix()
%% Calibrate camera
% Define images to process
path=['.' filesep 'Images' filesep];
list_imgs=dir([path '*.jpg']);
list_imgs_path=strcat(path,{list_imgs.name});

% Detect checkerboards in images
[imagePoints, boardSize, imagesUsed] = detectCheckerboardPoints(list_imgs_path);
imageFileNames = list_imgs_path(imagesUsed);

% Generate world coordinates of the corners of the squares
squareSize = 27;  % in units of 'mm'
worldPoints = generateCheckerboardPoints(boardSize, squareSize);

% Calibrate the camera
[cameraParams, imagesUsed, estimationErrors] = estimateCameraParameters(imagePoints, worldPoints, ...
    'EstimateSkew', true, 'EstimateTangentialDistortion', true, ...
    'NumRadialDistortionCoefficients', 3, 'WorldUnits', 'mm');
%% Compute homography for peripendicular plane to checkerboard
% Detect the checkerboard 
im=imread(['.' filesep 'Images' filesep 'exp_19.jpg']); %exp_19.jpg is the checkerboard orthogonal to the floor
[imagePoints, boardSize] = detectCheckerboardPoints(im);

% Compute rotation and translation of the camera.
[Rc, Tc] = extrinsics(imagePoints, worldPoints, cameraParams);

% Rc(rotation of the calibration view w.r.t the camera) = [x y z])
%then the floor has rotation Rf = [z x -y].(Normal vector of the floor goes up.)
Rf=[Rc(:,3),Rc(:,1),Rc(:,2)*-1];

% Translate it to the floor
H=452;%distance btw origin and floor
Fc = Rc * [0; H; 0];
Tc = Tc + Fc';

% Combine rotation and translation into one matrix:
Rf(3, :) = Tc;

% Compute the homography between the checkerboard and the image plane:
H = Rf * cameraParams.IntrinsicMatrix;

save('homographic_matrix.mat','H')
end

---

%% Transform points
function [x_transf,y_transf] =transform_points(xcor_i,ycor_i)
% creates a projective2D object and then transforms the points forward to
% get a top-view
% xcor_i and ycor_i are 1d vectors comprising of the x-coordinates and
% y-coordinates of trajectories. 
data=load('homographic_matrix.mat');
homo_matrix=data.H;
tform=projective2d(inv(homo_matrix));
[x_transf,y_transf] = transformPointsForward(tform,xcor_i,ycor_i);
end

---

Quoting text from OReilly Learning OpenCV Pg 412: "Once we have the homography matrix and the height parameter set as we wish, we could then remove the chessboard and drive the cart around, making a bird’s-eye view video of the path..." This what I essentially wish to achieve.

Answer 1

Abhishek,

I don't entirely understand what you are trying to do. Are your points on a plane, and are you trying to create a bird's eye view of that plane?

If so, then you need to know the extrinsics, R and t, describing the relationship between that plane and the camera. One way to get R and t is to place a checkerboard on the plane, and then use the extrinsics function.

After that, you can follow the directions in the question you cited to get the homography. Once you have the homography, you can create a projective2D object, and use its transformPointsForward method to transform your points.