How To Test If A 2d Point In Pygame Screen Is Part Of A 3d Object In Pyopengl?

I am making a game (RPG) in PyOpenGL which I have a crosshair. I want to check if a 3d object is in the crosshair (or detect if it is at a point), which is a 2d overlay. How can I

Solution 1:

What you see on the viewport is the 2 dimensional projection of a 3 dimensional scene. So each point on the 2D view port is a ray in the 3D scene which goes form near plane (near the eye) to the far plane. The object which is "seen" on the viewport is the first object which is "hit" by this ray.

The ray can be found with ease. See the answer to the question ray intersection misses the target.

To identify the object which is hit by this ray is hard. It strongly depends on the objects (meshs) which are drawn in your scene and can be achieve by Ray casting. You've to intersect each object (mesh) and to calculate the Euclidean distance to the intersection point. The object which is nearest to camera (eye) position is the "winner". How to intersect a ray and a object depends on the geometry and definition of the object.

---

Let me demonstrate this on an example. In the following I refer to the code of your previous question: How to rotate a certain object (Quad) in PyOpenGL?.

To find a ray through the world, you've to map window coordinates to object coordinates. If you've a crosshair in the middle of the screen, the the x and y window coordinates are

cross_x, cross_y = display[0]/2, display[1]/2

all points which have the same x and y coordinate are on the same ray, as seen from the camera position. The z coordinates of the 2 points on the ray are the minimum depth value (0) and the maximum depth value (1). To map window coordinates to object coordinates, gluUnProject can be used. The parameters to gluUnProject are of type GLdouble:

# get current view matrix, projection matrix and viewport rectangle 
mv_matrix = glGetDoublev(GL_MODELVIEW_MATRIX)
proj_matrix = glGetDoublev(GL_PROJECTION_MATRIX)
vp_rect = glGetIntegerv(GL_VIEWPORT)

# calculate "near" and "far" point 
pt_near = gluUnProject(cross_x, cross_y, 0, mv_matrix, proj_matrix, vp_rect)
pt_far  = gluUnProject(cross_x, cross_y, 1, mv_matrix, proj_matrix, vp_rect)

Add this code after

#Apply view matrix
glPopMatrix()
glMultMatrixf(viewMatrix)

If you've a circular object, then you've to intersect the ray with a sphere. Write a function which returns distance to the sphere if the ray intersect the sphere and None else. The algorithm of the following function I've taken from Peter Shirley's book Ray Tracing in One Weekend:

defsubtract(v0, v1):
    return [v0[0]-v1[0], v0[1]-v1[1], v0[2]-v1[2]]
defdot(v0, v1):
    return v0[0]*v1[0]+v0[1]*v1[1]+v0[2]*v1[2]
deflength(v):
    return math.sqrt(v[0]*v[0]+v[1]*v[1]+v[2]*v[2])
defnormalize(v):
    l = length(v)
    return [v[0]/l, v[1]/l, v[2]/l]

# Ray - Sphere intersection## Sphere:         dot(p-C, p-C) = R*R    `C`: center, `p`: point on the sphere, `R`, radius # Ray:            p(t) = A + B * t       `A`: origin, `B`: direction        # Intersection:   dot(A+B*t-C, A+B*t-C) = R*R#                 t*t*dot(B,B) + 2*t*dot(B,A-C) + dot(A-C,A-C) - R*R = 0defisectSphere(p0, p1, C, R):
    A = p0               # origin 
    B = normalize(subtract(p1, p0)) # direction
    oc = subtract(A, C) 
    a = dot(B, B)
    b = 2 * dot(oc, B)
    c = dot(oc, oc) - R*R
    discriminant = b*b - 4*a*c
    if discriminant > 0:
        t1 = (-b - math.sqrt(discriminant)) / (2*a)
        t2 = (-b + math.sqrt(discriminant)) / (2*a)
        t = min(t1, t2)
        return t if t >= 0.0elseNonereturnNone

Use the function somehow as follows:

dist = isectSphere(pt_near, pt_far, person.pos, 1.0)

if dist != None:
    print(dist)
else:
    print("no hit")

---

---

The intersection with an axis aligned cuboid is takes much more effort. A cuboid has 6 sides. You have to intersect each side and to find the which is closest. Each side is a quad. The intersection with a quad can be composed of 2 triangles.

For intersecting a ray and a triangle, i've ported the code of the answer to the question How to identify click inside the 3D object or outside 3D object using near and far positions from c++ to python:

defmults(v, s):
    return [v[0]*s, v[1]*s, v[2]*s]
defadd(v0, v1):
    return [v0[0]+v1[0], v0[1]+v1[1], v0[2]+v1[2]]
defcross(v0, v1):
    return [
        v0[1]*v1[2]-v1[1]*v0[2],
        v0[2]*v1[0]-v1[2]*v0[0],
        v0[0]*v1[1]-v1[0]*v0[1]]

defPointInOrOn( P1, P2, A, B ):
    CP1 = cross( subtract(B, A), subtract(P1, A) )
    CP2 = cross( subtract(B, A), subtract(P2, A) )
    return dot( CP1, CP2 ) >= 0defPointInOrOnTriangle( P, A, B, C ):
    return PointInOrOn( P, A, B, C ) and PointInOrOn( P, B, C, A ) and PointInOrOn( P, C, A, B )

# p0, p1   points on ray# PA, PB, PC  points of the triangledefisectPlane(p0, p1, PA, PB, PC):
    R0 = p0               # origin 
    D = normalize(subtract(p1, p0))
    P0 = PA
    NV = normalize( cross( subtract(PB, PA), subtract(PC, PA) ) )
    dist_isect = dot( subtract(P0, R0), NV ) / dot( D, NV ) 
    P_isect    = add(R0, mults(D, dist_isect))
    return P_isect, dist_isect
defisectTrianlge(p0, p1, PA, PB, PC):
    P, t = isectPlane(p0, p1, PA, PB, PC)
    if t >= 0and PointInOrOnTriangle(P, PA, PB, PC):
        return t
    returnNone

The intersection of a quad instead of a triangle is similar:

defPointInOrOnQuad( P, A, B, C, D ):
    return (PointInOrOn( P, A, B, C ) and PointInOrOn( P, B, C, D ) and
            PointInOrOn( P, C, D, A ) and PointInOrOn( P, D, A, B ))

defisectQuad(p0, p1, PA, PB, PC, PD):
    P, t = isectPlane(p0, p1, PA, PB, PC)
    if t >= 0and PointInOrOnQuad(P, PA, PB, PC, PD):
        return t
    returnNone

For the intersection with a cuboid, the intersection with the closets side has to be found in a loop. The cuboid is defined by the 2 points on the diagonal across its volume:

defisectCuboid(p0, p1, pMin, pMax):
    t = Nonetry:
        pl = [[pMin[0], pMin[1], pMin[2]], [pMax[0], pMin[1], pMin[2]],
              [pMax[0], pMax[1], pMin[2]], [pMin[0], pMax[1], pMin[2]],
              [pMin[0], pMin[1], pMax[2]], [pMax[0], pMin[1], pMax[2]],
              [pMax[0], pMax[1], pMax[2]], [pMin[0], pMax[1], pMax[2]]]
        il = [[0, 1, 2, 3], [4, 5, 6, 7], [4, 0, 3, 7], [1, 5, 6, 2], [4, 3, 1, 0], [3, 2, 6, 7]]
        for qi in il:
            ts = isectQuad(p0, p1, pl[qi[0]], pl[qi[1]], pl[qi[2]], pl[qi[3]] )
            if ts != Noneand ts >= 0and (t == Noneor ts < t):
                t = ts
    except:
        t = Nonereturn t

The cuboid is moved through the scene so it is possible to define its position in the world. But its orientation changes dynamically too, because of the rotation. So the cuboid is not axis aligned in world space, but it is axis aligned in object space. This means the points of the ray have to be transformed to object space rather than world space. The object space matrices are set after the model transformation in the .draw() method of the cuboid. Move the intersection test to the .draw() method:

classPerson:

    # [...]defdraw(self):
        global dist

        glTranslated(self.pos[0], self.pos[1], self.pos[2])
        glRotated(self.rot,0,0,1)

        mv_matrix = glGetDoublev(GL_MODELVIEW_MATRIX)
        proj_matrix = glGetDoublev(GL_PROJECTION_MATRIX)
        vp_rect = glGetIntegerv(GL_VIEWPORT)

        cross_x, cross_y = display[0]/2, display[1]/2
        pt_near = gluUnProject(cross_x, cross_y, 0, mv_matrix, proj_matrix, vp_rect)
        pt_far = gluUnProject(cross_x, cross_y, 1, mv_matrix, proj_matrix, vp_rect)

        #dist = isectSphere(pt_near, pt_far, [0, 0, 0], 1.0)
        dist = isectCuboid(pt_near, pt_far, [-1, 0, -1], [1, 1, 1])

        if dist != None:
            print(dist)
        else:
            print("no hit")

        glBegin(GL_QUADS) #Begin fillfor surface in self.surfaces:
            for vertex in surface:
                glColor3f(0,1,0)
                glVertex3fv(self.vertices[vertex])
        glEnd()
        glLineWidth(5) #Set width of the line
        glBegin(GL_LINES) #Begin outlinefor edge in self.edges:
            for vertex in edge:
                glColor3f(1,1,0)
                glVertex3fv(self.vertices[vertex])
        glEnd()