Unproject mouse to get 3D world coordinates Libgdx

Question

My Question: How does one get a 3D model to move with the mouse cursor keeping the y location of the model at 0. (Using Libgdx)

What I have tried: I'm trying to get my 3D models to follow my cursor.

At the moment I have simply made the models move with the x and y coordinates of my mouse with adding factors like a multiplier and camera location. This is not very good because it doesn't follow the cursor and if I rotate the camera then the model doesn't move in the correct direction.

This is the code I created for moving the model :

instances.get(modelIndex).transform.setToTranslation(((Gdx.input.getX()-650)/10) + cam.position.x,0,((Gdx.input.getY()-400)/10)+ cam.position.z);

Here's a link to an image of the problem (The green circle is where the mouse is).

I know that I'm going about this wrong and that I should be using the unproject method. I have tried to use the unproject method and did have little success, the model was just stuck to the near pane of the camera. I want my object to stay at 0 on the y axes so I used this:

cam.unproject(mousePos);
instances.get(modelIndex).transform.setToTranslation(mousePos.x, 0, mousePos.z);

This then resulted in the model moving quite slowly(not with the mouse), in the right direction. Also when I rotated or moved the camera the model would be set somewhere in the distance or even behind the camera.

Answer 1

You cannot simply use camera.unproject, because a single 2D coordinate can represent multiple coordinates in 3D space. Instead you can use camera.getPickRay to get the Ray, which is defined by the origin (the location on the camera's near plane) along with a direction into 3D space. So, you'll get the line in 3D space that represents the 2D coordinate:

Ray ray = cam.getPickRay(screenX, screenY);

Now you want the point (distance from origin) on that line (ray) where y=0:

ray.origin.y + ray.direction.y * distance = 0 <=>
ray.direction.y * distance = -ray.origin.y <=>
distance = -ray.origin.y / ray.direction.y

Resulting in:

float distance = -ray.origin.y / ray.direction.y;

Now to get the position on the ray at that distance:

Vector3 tmpVector = new Vector3();
tmpVector.set(ray.direction).scl(distance).add(ray.origin);

To set the model instance translation to that value you can use:

instance.transform.setTranslation(tmpVector);

Here instance is referring to your instances.get(modelIndex).

I would encourage you to use InputProcessor (see http://code.google.com/p/libgdx/wiki/InputEvent), so to put it all together:

Vector3 tmpVector = new Vector3();
@Override
public boolean touchDragged (int screenX, int screenY, int pointer) {
    Ray ray = cam.getPickRay(screenX, screenY);
    final float distance = -ray.origin.y / ray.direction.y;
    tmpVector.set(ray.direction).scl(distance).add(ray.origin);
    instance.transform.setTranslation(tmpVector);
    return true;
}

Note that you might need to override touchdown and return true for touchDragged to be called.