How can I check if an object(s) are in front of the camera?

Question

I have got some trees, which are greatly lagging the game, so I would like to check if the trees are in front of the camera or not.

I have had some help from the Mathematics forum, and also had a look at This link to help me convert pitch/yaw to the directional vector needed.

But for some reason, whenever I move the camera to the left, the trees become visible, wheras whenever I move it to the right, they become unvisible (So if camera is pointing at +1 on the Z axis, it seems to be rendering the trees, but -1 on the Z axis and it seems to not render them). (See http://i.gyazo.com/cdd05dc3f5dbdc07577c6e41fab3a549 for a less-jumpy .mp4)

I am using the following code to check if an object is in front of the camera or not:

Ship you = shipsID.get(UID);
int dis = 300;
Vector3f X = new Vector3f(camera.x(), camera.y(), camera.z());
float x = (float) (Math.cos(Math.toRadians(camera.yaw()))*Math.cos(Math.toRadians(camera.pitch())));
float y = (float) (Math.sin(Math.toRadians(camera.yaw()))*Math.cos(Math.toRadians(camera.pitch())));
float z = (float) Math.sin(Math.toRadians(camera.pitch()));
Vector3f V = new Vector3f(x, y, z);

for (Tree tree : trees){
    Vector3f Y = new Vector3f(tree.location.x, tree.location.y, tree.location.z);
    Vector3f YMinusX = Y.negate(X);//new Vector3f(Y.x - X.x, Y.y - X.y, Y.z - X.z);
    float dot = Vector3f.dot(YMinusX, V);
    if (dot > 0){
        tree.render();
    }
}

Is anyone able to tell me what I have done wrong here? I can't work out if it's the math.. Or the code.. Or what?

Camera translation code:

 public void applyTranslations() {
    glPushAttrib(GL_TRANSFORM_BIT);
    glMatrixMode(GL_MODELVIEW);
    glRotatef(pitch, 1, 0, 0);
    glRotatef(yaw, 0, 1, 0);
    lastYaw = yaw;
    glRotatef(roll, 0, 0, 1);
    glTranslatef(-x, -y, -z);
    glPopAttrib();
}

UPDATE:

It appears to be where the camera is looking. For example, if I look to -Z, nothing happens, but if I look to +Z, they all render. The if (dot > 0) code appears to somehow being +Z rather than +TheCameraRotation.

Answer 1

Your camera rotations yaw around Y, implying Y is your up vector. However, float z = (float) Math.sin(Math.toRadians(camera.pitch())); gives Z for your up vector. There is an inconsistency. I'd start by swapping y and z here, then print everything out every frame so you can see what happens as you rotate the camera. Also render just one tree and print dot. E.g. you might quickly notice the numbers approach 1.0 only when you look at 90 degrees left of the tree which narrows down the problem. As @DWilches notes, swapping cos/sin will change the phase of the rotation, which would produce such an effect.

You might consider limiting the dot product to the camera's field of view. There are still problems in that trees are not just points. A better way would be to test tree bounding boxes against the camera frustum, as @glampert suggests.

Still, the tree geometry doesn't look that complex. Optimization wise, I'd start trying to draw them faster. Are you using VBOs? Perhaps look at methods to reduce draw calls such as instancing. Perhaps even use a few models for LOD or billboards. Going even further, billboards with multiple trees on them. Occlusion culling methods could be used to ignore trees behind mountains.

[EDIT]
Since your trees are all roughly on a plane, you could limit the problem to the camera's yaw:

float angleToTree = Math.atan2(tree.location.z - camera.z(), tree.location.x - camera.x());
float angleDiff = angleToTree - camera.yaw();
if (angleDiff > Math.PI)
    angleDiff -= 2.0f * Math.PI;
if (angleDiff < -Math.PI)
    angleDiff += 2.0f * Math.PI;
if (abs(angleDiff) < cameraFOV + 0.1f) //bias as trees are not points
    tree.render();