OpenCV project 3D coordinates to 2d camera coordinates

Question

I have the 3D-world coordinates of an object and I want to get its coordinates in the camera-2D-plane. I have already calibrated the camera using cv::calibrateCamera, so that I have the camera matrix and distortionCoeffs.

For projecting the 3D-point to 2d-camera-coordinates, I use cv::projectPoints. Documentation says:

void projectPoints(InputArray objectPoints, InputArray rvec, InputArray tvec, InputArray cameraMatrix, InputArray distCoeffs, OutputArray imagePoints, OutputArray jacobian=noArray(), double aspectRatio=0 )

How do I get rvec/ tvec and is my approach right?calibrateCameragives me rvecs and tvecs, but they are for each input chessboard-image and I guess, they are the rotation and translation of the chessboard and not of the camera.

Answer 1

As I have the projection matrix (P), I can calculate the 2D coordinates (x) in the camera plane like this from the 3D-coordinates X (using homogeneous coordinates):

x (u,v,w) = P * X (x1,x2,x3,w)

I just have a problem with the rectification, so that I do not get the exact projection to my 2d image. See here: OpenCV stereo vision 3D coordinates to 2D camera-plane projection different than triangulating 2D points to 3D