Reprojection error

The reprojection error is a geometric error corresponding to the image distance between a projected point and a measured one. It is used to quantify how closely an estimate of a 3D point [math]\displaystyle{ \hat{\mathbf{X}} }[/math] recreates the point's true projection [math]\displaystyle{ \mathbf{x} }[/math]. More precisely, let [math]\displaystyle{ \mathbf{P} }[/math] be the projection matrix of a camera and [math]\displaystyle{ \hat{\mathbf{x}} }[/math] be the image projection of [math]\displaystyle{ \hat{\mathbf{X}} }[/math], i.e. [math]\displaystyle{ \hat{\mathbf{x}}=\mathbf{P} \, \hat{\mathbf{X}} }[/math]. The reprojection error of [math]\displaystyle{ \hat{\mathbf{X}} }[/math] is given by [math]\displaystyle{ d(\mathbf{x}, \, \hat{\mathbf{x}}) }[/math], where [math]\displaystyle{ d(\mathbf{x}, \, \hat{\mathbf{x}}) }[/math] denotes the Euclidean distance between the image points represented by vectors [math]\displaystyle{ \mathbf{x} }[/math] and [math]\displaystyle{ \hat{\mathbf{x}} }[/math].

Minimizing the reprojection error can be used for estimating the error from point correspondences between two images. Suppose we are given 2D to 2D point imperfect correspondences [math]\displaystyle{ \{\mathbf{x_i} \leftrightarrow \mathbf{x_i}'\} }[/math]. We wish to find a homography [math]\displaystyle{ \hat{\mathbf{H}} }[/math] and pairs of perfectly matched points [math]\displaystyle{ \hat{\mathbf{x_i}} }[/math] and [math]\displaystyle{ \hat{\mathbf{x}}_i' }[/math], i.e. points that satisfy [math]\displaystyle{ \hat{\mathbf{x_i}}' = \hat{H}\mathbf{\hat{x}_i} }[/math] that minimize the reprojection error function given by

[math]\displaystyle{ \sum_i d(\mathbf{x_i}, \hat{\mathbf{x_i}})^2 + d(\mathbf{x_i}', \hat{\mathbf{x_i}}')^2 }[/math]

So the correspondences can be interpreted as imperfect images of a world point and the reprojection error quantifies their deviation from the true image projections [math]\displaystyle{ \hat{\mathbf{x_i}}, \hat{\mathbf{x_i}}' }[/math]

References

• Richard Hartley and Andrew Zisserman (2003). Multiple View Geometry in computer vision. Cambridge University Press. ISBN 0-521-54051-8.