Intersection of camera ray and 3D plane

Suppose we have a camera in 3D space and want to know where a ray emanating from a selected pixel intersects with a 3D plane. To solve this you’ll need to know the camera’s intrinsic properties (eg. focal length). It’s handy if you have the commonly used 3×3 camera intrinsic matrix available(usually from calibration), which allows you to convert from image points to normalised points as follows:

\mathbf{p_{norm}} = K^{-1}\mathbf{p}

where, K is the 3×3 intrinsic matrix and p=[x, y, 1] is the image point, expressed as a homogeneous vector. The normalisation does the following:

  • moves the origin (0,0) to the centre of the image (principal point)
  • focal length becomes 1 unit

Using normalised image points, our problem can be illustrated as shown below, where \left(x_i,\,y_i\right) are the normalised image points.

A ray coming from the camera can be described by a line vector. The line vector is just a vector from the origin (0,0,0) to the normalised image point in 3D space, the vector is just \left(x_i,\, y_i,\, 1\right).  To fully describe the line, we write it in parametric form and introduce the variable t:

Ray(t) = \left(x_{i}t,\,y_{i}t,\,t\right)

Note: x_{i} and y_{i} are constants

Now a 3D plane in space (of infinite size) can be described by the plane equation:

Ax + By + Cz + D = 0

Combining the line and plane equation and solving for t gives:

Ax_{i}t + By_{i}t + Ct + D = 0

t =\frac{-D}{Ax_{i} + By_{i} + C}

Substitute t back into Ray(t) to find the 3D point of intersection.

53 thoughts on “Intersection of camera ray and 3D plane”

  1. I have an augmented square in space. How do I use the square with it being in the camera image space and all of its features distorted?

  2. I calculated the intrinsic matrix using openCV’s chessboard algorithm. I assume this is K you are talking about. How do I actually multiply the points(P) – which are 4 – with K? Should I store them how?

  3. I need to know (3) items that your great blog brings to mind. First, does your line require a 90 degree incidence with the plane? Would like to know intersection point of line from camera but with non-90 angle of incidence. Second, following up on term, I too calibrated the square with an output of 5 distortion coefficients. Leaving them out vastly – making them all 0’s – improved my project points in openCV. I chose a very low distortion lens – would this contribute? That doesn’t seem quite right. Finally, manipulating the plane in openCV requires 4 + points yet you are somehow getting away with (3)! 🙂 I know its the calculus – but – do you have any trades offs to using it with only 3 points.

    Thanks for your contribution to this area!

    1. The line can be at any angle. What’s the context of your 2nd and 3rd question? It doesn’t seem related to this article.

      1. Sorry I have returned to the blog so late, I don’t think my email is giving me proper notification. I have self-answered item 2. With regard to question 3 does opencv provide the groundwork to working towards your same solution here? I have successfully implemented the solve PnP solution which successfully produces camera pose and world coordinate generation but requires 4 coplanar points rather than 3 and no ray targeting in doing so.

  4. Also how did you define your plane’s corner points for 3D solution? Did you use a x,y model with z = 0 for example?

    Best

    1. In this article the plane is assumed to be infinitely large when using the Ax + By + Cz + D = 0 equation, so there are no finite corners.

      1. Now a 3D plane in space (of infinite size) can be described by the plane equation:

        Ax + By + Cy + D = 0

        Did you mean Cz?

  5. It didn’t post all of my math : (
    try again:
    My line vector is: +t
    Distribute t: +
    Make parametric eqs:
    x = 0+txi, y = 0+tyi, z = 0+t1
    x = txi, y = tyi, z =t1
    My plane is a special case(Cz = 0 where A & B = 0)
    How do I get to t = -D/Axi(=0)Byi(=0)Cz
    t = -D/Cz
    It seems I have two unknowns?

  6. No luck. I did use the parametric forms of the line origin 0,0,0 to (xi, yi, 1) and equation 0x + 0y + -550000z = 0

      1. https://www.math24.net/plane/
        A = B = 0
        see Special cases of this equation of a plane
        #3
        Good point but maybe instead of non-valid but special case.
        Finding the plane equation with 3 planar points that make 2 vectors: both in the Z-plane result in – normals from orthogonal line to plane. Can post example –

  7. Ok, so I am normalizing image points by K-1(inverse) from camera calibration. How do I modify image point (Xi , Yi)
    to vector so I can multiply by inverse K to normalize? Use (Xi, Yi, 1)?

  8. Can you please comment on your normalization procedure to get your results (xi, yi, 1);

    My hardware is rendering several image points but when I multiply the inverse of K by a raw image point I am getting crazy answers for normalization. I think its something simple.

    1. There’s not much to it other than inv(K) * [x; p; 1], where (x,y) are the raw image co-ordinates.

  9. Input Raw Image Testpoints x, y, z:
    [50, 60, 1]
    invK:[0.002095520423466527, 0, -0.8025476036257788;
    0, 0.002096599465309339, -0.4776608362190569;
    0, 0, 1]
    Normalized coordinates = invK *[50, 60, 1]
    Pnorm = [0.1047760211733263, 0.1257959679185603, -67.78703035443236]

      1. Awesome, thank you! That got it after I looked closely at your response. What units are the normalized points in? Shouldn’t they be multiplied by 1000 (guessing here)?

        Confirm same result:
        Input Raw Image Testpoints x, y, z:
        [50;
        60;
        1]
        invK:[0.002095520423466527, 0, -0.8025476036257788;
        0, 0.002096599465309339, -0.4776608362190569;
        0, 0, 1]
        Normalized coordinates = invK *[50;
        60;
        1]
        Pnorm = [-0.6977715824524525;
        -0.3518648683004966;
        1]

          1. Thank you for your help so far. I’ve gotten to the end of this. But I am not understanding correctly:

            “Substitute t back into Ray(t) to find the 3D point of intersection”. Take t = -D/ Axi + Byi + C and put it where on the Ray(t) equation?

  10. When do we use normalized coordinates? Do we use them with raw, then line normal to plane, for vectors used in vector cross and in other functions utilizing raw image point data?

    1. Various computer vision problems make use of normalized coordinates, this is one of them. Just depends on the problems.

  11. Ok, so it turns out my image plane and model plane are the same z = 1, right on top of each other. Should I move my model plane back to gain some space for the ray? Also how do you back convert the equation intersect point Pnorm(xi, yi, 1) to equal image coordinates? pNorm * K?

    1. The math should still works out if the image/model planes are on top of each other. To convert normalize back to pixel space do K * pNorm.

  12. Hi,

    I just stumbled on the rest of your site. It is clear you like teaching. Had I known of the rest of your site I probably wouldn’t have had so many on this one blog : ) I am going to redirect some questions in the Q & A section. Thx

  13. I’ve been returning to your articles frequently(sorry not more consistently). I have a simple question:

    How do I solve for D in your equation

    t = -D/(Axi+Byi+Cz)

  14. Hello,
    Thanks for this great article.
    I stumbled on this why struggling to successfully implement Inverse Perspective Mapping from a 2D Image.

    I can calculate inv(K) * (u,v,1). If I want to assume all my pixel points lie on the ground plane and the ground plane is flat (z=0), Is this a valid equation?
    Because all a,b,d = 0 in this and c=1. This makes t=0 according to your equation. Can you kindly help how to solve this type of flat ground plane assumption? Thank you so much in advance.

    1. The coefficient A, B, C represents the [x, y, z] normal of the plane. So if the ground is flat at z=0, the normal is [0, 0, 1]. However be careful, the ground plane has to be relative to the camera’s co-ordinate system. The camera has +y into the ground and -y to the sky. So the plane is really at [0, -1, 0], assuming the camera is looking forward (no tilt). The coefficient D is the height relative to the camera. I might make a tutorial detailing this specific scenario, thanks for the idea.

  15. Does your method retain accuracy when the image plane has a projected plane that is distorted by perspective effects like when the plane on the image is rotated a bit.

      1. That is if the plane equation is based on the undistorted plane instead of image points by themselves. For example the plane derived from solvePnP with correspondences to a model. Get the plane eq from the model. Let me know if I got this wrong.

        So if that is right how do I compute the intersecting line to plane from the principle axis of the camera? You have a lot of great info on creating the line but I don’t see it as a ray from camera coordinates/camera pose.

        1. With solvePnP you can find the 3D points of your plane, relative to the camera. Using these 3D points you can then find the plane equation.

  16. Some of my Pnorm values are (-). What does this mean?
    eg, [x,y,z]:
    [0.383621976759394, -0.2925051609396598, 1;]
    [ -0.5374948605305616, -0.3194088054650904,1];

  17. Hopefully these may be last questions. You have been very patient with this, thank you. : ) : )
    I reviewed the axis in post diagram.
    What do the normalized points refer to in the image plane — pixel space?

    1. It’s in normalized space with (0,0,1) at the center of the image plane. So x left is negative, x right is positive. The normalization when you multiply by inv(K) is really x’ = (x – center_x)/focal_x.

Leave a Reply

Your email address will not be published. Required fields are marked *