gluProject Converting 3D coordinates to 2D coordinates does not convert the 2D Y Coordinate correctly

Question

After two hours of googling (here, here, here, here, and here, and a ton others which I am not bothered to find), I thought I had finally learnt the theory of turning 3D coordinates to 2D coordinates. But it isn't working. The idea is to translate the 3D coordinates of a ship to 2D coordinates on the screen to render the username of the player controlling that ship.

However, the text is rendering in the wrong location:

The text is "Test || 2DXCoordinate || 2DZCoordinate".

Here is my getScreenCoords() - Which converts the 3D coordinates to 2D.

public static int[] getScreenCoords(double x, double y, double z) {
    FloatBuffer screenCoords = BufferUtils.createFloatBuffer(4);
    IntBuffer viewport = BufferUtils.createIntBuffer(16);
    FloatBuffer modelView = BufferUtils.createFloatBuffer(16);
    FloatBuffer projection = BufferUtils.createFloatBuffer(16);
    GL11.glGetFloat(GL11.GL_MODELVIEW_MATRIX, modelView);
    GL11.glGetFloat(GL11.GL_PROJECTION_MATRIX, projection);
    GL11.glGetInteger(GL11.GL_VIEWPORT, viewport);
    boolean result = GLU.gluProject((float) x, (float) y, (float) z, modelView, projection, viewport, screenCoords);
    if (result) {
        return new int[] { (int) screenCoords.get(0), (int) screenCoords.get(1) };
    }
    return null;
}

screenCoords.get(0) is returning a perfect X coordinate. However, screenCoords.get(1) is going higher or lower depending on how far away I am from the ship. After many hours of debugging, I have narrowed it down to this line being incorrect:

GLU.gluProject((float) x, (float) y, (float) z, modelView, projection, viewport, screenCoords);

However, I have no idea what is wrong. The X coordinate of the ship is fine.... Why not the Y?

According to BDL's answer, I am supplying the "wrong matrix" to gluProject(). But I don't see how that is possible, since I call the method right after I render my ship (Which is obviously in whatever matrix draws the ship).

I just can't fathom what is wrong.

Note: BDL's answer is perfectly adequate except that it does not explain why the Y coordinates are incorrect.

Note: This question used to be much longer and much more vague. I have posted my narrowed-down question above after hours of debugging.

Answer 1

You have to use the same projection matrix in gluProject that you use for rendering your ship. In your case the ship is rendered using a perspective projection, but when you call gluProject a orthographic projection is used.

General theory about coordinate systems in OpenGL

In most cases geometry of a model in your scene (e.g. the ship) is given in a model-coordinate system. This is the space where your vertex coordinates exist. When now placing the model in your scene we apply the model-matrix to each vertex to get the coordinates the ship has in the scene. This coordinate system is called world space. When viewing the scene from a given viewpoint and a viewing direction, again a transformation is needed that transforms the scene such that the viewpoint is located in the origin (0,0,0) and view-direction is along the negativ z-axis. This is the view coordinate system. The last step is to transform view-coordinates into ndc, which is done via a projection matrix.

In total we get the transformation of a vertex to the screen as:

 v_screen = Projection * View * Model * v_model

In ancient OpenGL (as you use it) View and Model are stored together in the ModelView matrix.

(I skipped here some problems as perspective divide, but it should be sufficient to understand the problem.)

Your problem

You already have a position in world space (x,y,z) of your ship. Thus the transformation with Model has already happend. What is left is

v_screen = Projection * View * v_worldspace

For this we see, that in our case the ModelView matrix that gets entered to gluProject has to be exactly the View matrix.

I can't tell you where you get the view matrix in your code, since I don't know this part of your code.