Suppose we have a camera in 3D space and want to know where a ray emanating from a selected pixel intersects with a 3D plane. To solve this you’ll need to know the camera’s intrinsic properties (eg. focal length). It’s handy if you have the commonly used 3×3 camera intrinsic matrix available(usually from calibration), which allows you to convert from image points to normalised points as follows:
where, K is the 3×3 intrinsic matrix and p are the image points, expressed as a 3×1 matrix of homogeneous points. The normalisation does the following:
- moves the origin (0,0) to the centre (or close to) of the image
- focal length becomes 1 unit
Using normalised image points, our problem can be illustrated as shown below, where are the normalised image points.
A ray coming from the camera can be described by a line vector. The line vector is just a vector from the origin (0,0,0) to the normalised image point in 3D space, the vector is just . To fully describe the line, we write it in parametric form and introduce the variable t:
Note: and are constants
Now a 3D plane in space (of infinite size) can be described by the plane equation:
Ax + By + Cy + D = 0
Combining the line and plane equation and solving for t gives:
Substitute t back into Ray(t) to find the 3D point of intersection.