Interpreting the Reprojection Error from camera calibration

Question

I'm kind of confused about the Reprojection Error during camera calibration. I understood that the Reprojection Error describes the differences between the detected point and the world point. I even found out that a value <1 is "good".

But what does it tell? I mean e.g. the Reprojection Error is 2: That means the distance is 2px, so far so good. But what does it mean in reference to the calibration? Is a calibration required? or will this value get adjusted by the calibration process to 0?

To be more general: What does the value causes/tells us?

Answer 1

From Mathworks:

Reprojection errors provide a qualitative measure of accuracy. A reprojection error is the distance between a pattern keypoint detected in a calibration image, and a corresponding world point projected into the same image. The showReprojectionErrors function provides a useful visualization of the average reprojection error in each calibration image. If the overall mean reprojection error is too high, consider excluding the images with the highest error and recalibrating.

Think about it like this. Let's say you're calibrating a face. Your model assumes each of the keypoints of your face to be a certain proportion apart. Of course, each photo will have slightly different different proportional differences than your model. And your model takes like 100 of them, and averages them out to determine the "average" proportions of the face. But what if 10 of those pictures were at weird angles, or were distorted somehow. They are outliers, and they might be throwing off your model. Perhaps it's better to exclude them from your model calculation so you can get a more normalized model of what a normal face looks like. You can tell what is whacked out by looking at the reprojection errors.