How can I detect if a point is inside a cone or not, in 3D space?

Question

How is possible to detect if a 3D point is inside a cone or not?

Ross cone = (x1, y1, h1)
Cone angle = alpha
Height of the cone = H
Cone radius = R
Coordinates of the point of the cone = P1 (x2, y2, h2)
Coordinates outside the cone = P2( x3, y3, h3)

Result for point1 = true
Result for point2 = false

Answer 1

To expand on Ignacio's answer:

Let

x = the tip of the cone
dir = the normalized axis vector, pointing from the tip to the base
h = height
r = base radius

p = point to test

So you project p onto dir to find the point's distance along the axis:

cone_dist = dot(p - x, dir)

At this point, you can reject values outside 0 <= cone_dist <= h.

Then you calculate the cone radius at that point along the axis:

cone_radius = (cone_dist / h) * r

And finally calculate the point's orthogonal distance from the axis to compare against the cone radius:

orth_distance = length((p - x) - cone_dist * dir)

is_point_inside_cone = (orth_distance < cone_radius)

Answer 2

The language-agnostic answer:

• Find the equation of the line defining the main axis of your cone.
• Compute the distance from the 3D point to the line, along with the intersection point along the line where the distance is perpendicular to the line.
• Find the radius of your cone at the intersection point and check to see if the distance between the line and your 3D point is greater than (outside) or less than (inside) that radius.