Off-axis projection with glFrustum

Question

I am trying to do an off-axis projection of a scene with OpenGL and I gave a read to the document to Robert Kooima's off-axis projection and have a much better idea now of what actually has to be done but there are still some pieces which I am finding tricky here. I got to know of the off-axis projection code for OpenGL to be somewhat as follows:

Code 1:

glMatrixMode(GL_PROJECTION);  
    glLoadIdentity();            
    glFrustum(fNear*(-fFov * ratio + headX),  
              fNear*(fFov * ratio + headX),  
              fNear*(-fFov + headY),  
              fNear*(fFov + headY),  
              fNear, fFar);  
          
    glMatrixMode(GL_MODELVIEW);  
    glLoadIdentity();  
    gluLookAt(headX*headZ, headY*headZ, 0, headX*headZ, headY*headZ, -1, 0, 1, 0);
    glTranslatef(0.0,0.0,headZ);

Had this been a normal perspective projection with the user at the center of the screen, it is fairly easy to understand as I comprehend.

               Screen  
                   |
                   |  h = H/2
                   |  
x----- n -----------
                   |
                   |  h = H/2
                   |

With the user at x and distance from screen being n, the top, bottom coordinates for glFrustum would be calculated as: (assume theta is the Field of View (fov) which I suppose is assumed as 30 degrees)

h = n * tan (theta/2);
tanValue = DEG_TO_RAD * theta/2;
[EDIT Line additon here>>]: fFov = tan(tanValue);
h = n * tan (tanValue);

Hence, top and bottom (negating top's value) are both obtained for glFrustum arguments. Left one's are left/right for now.

Now, Aspect Ratio, r = ofGetWidth()/ofGetHeight();
Right = n * (fFov * r); , where r is the aspect ratio [Edit1>> Was written tanValue*r earlier here]

Question 1) Is the above (tanValue*r) getting the horizontal fov angle and then applying the same to get left/right value?

   double msX = (double)ofGetMouseX();
   double msY = (double)ofGetMouseY();
   double scrWidth = (double)ofGetWidth();
   double scrHeight = (double)ofGetHeight();

   headX = (msX / scrWidth) - 0.5;
   headY = ((scrHeight - msY) / scrHeight) - 0.5;
   headZ = -2.0;

Now, consider the projection off-axis and we have the headX and headY position computed (using mouse here instead of actual user's head):

Question 2) How is the headX and y being computed and what is the use subtracting -0.5 from the above? I observed that it brings the x-value to (-0.5 to 0.5) and y-value to (0.5 to -0.5) with msX and msY varying.

Question 3) In the above code (Code 1), how is headY being added to the calculated to the tan(fov/2) value?

-fFov + headY
fFov + headY

What does this value provide us with? -fFov was the calculated tan of theta/2 but how can headY be added to directly?

-fFov * ratio + headX
-fFov * ratio + headX

How does the abvoe give us a vlaue which wehn multiplied by n (near value) gives us left and right for the assymetric glFrustum call for off-axis projection?

Question 4) I understand that the glLookAt has to be done for View Point to shift the apex of the frustum to where the eye of the user is (in this case where the mouse is). Notice the line in the above code:

gluLookAt(headX*headZ, headY*headZ, 0, headX*headZ, headY*headZ, -1, 0, 1, 0);

How is headX*headZ giving me the xPosition of the eye, headY*headZ giving me the yPosition of the eye which I can use in gluLookAt() here?

EDIT: Full problem description added here: pastebin.com/BiSHXspb

Answer 1

You have made this nice picture of ASCII art

               Screen  
                   B
                   |  h = H/2
                   |  
x----- n ----------A
                   |
                   |  h = H/2
                   B'

The field of view is defined as the angle fov = angle((x,B), (x,B')) formed between the two tips B, B' of the screen "line" and the point x. The trigonometric function Tangens (tan) is defines as

h/n = tan( angle((x,A), (x,B)) )

And since length(A, B) == length(A, B') == h == H/2 we know that

H/(2·n) == tan( fov ) == tan( angle((x,B), (x,B')) ) == tan( 2·angle((x,A), (x,B)) )

Since in trigonometry angles are given in radians, but most people are more comfortable with degrees you may have to convert from degress to radians.

So we're interested in only half of the screen span (= h) we've to half the angle. And if we want to accept degress also convert it to radians. That's what this expression is meant for.

tanValue = DEG_TO_RAD * theta/2;

Using that we then calculate h by

h = tan(tanValue) * n

If the FOV is for horizontal or vertical span of the screen depends on the way how the field span H is scaled with the aspect ratio.

How is the headX and y being computed and what is the use subtracting -0.5 from the above? I observed that it brings the x-value to (-0.5 to 0.5) and y-value to (0.5 to -0.5) with msX and msY varying.

The calculations you gave assume that screen space coordinates are in a range [0, screenWidth] × [0, screenHeight]. However since we're doing our frustum calculations in a normalized range [-1, 1]² we want to bring the device absolute mouse coordinates to normalized center relative coordinates. This allows then to specify the axis offset relative to the normalized near plane size. This is how it looks with 0 offset (the grid has 0.1 units distance in this picture):

And with a X offset of -0.5 applied it looks like this (orange outline), as you can see the left edge of the near plane has been shifted to -0.5.

Now simply imagine that the grid was your screen, and your mouse pointer would drag around the projection frustum near plane bounds like that.

What does this value provide us with? -fFov was the calculated tan of theta/2 but how can headY be added to directly?

Because fFov is not an angle but the span H/2 = h in your ASCII art picture. And headX and headY are relative shifts in the normalized near projection plane.

How is headXheadZ giving me the xPosition of the eye, headYheadZ giving me the yPosition of the eye which I can use in gluLookAt() here?

The code you're quoted seems to be an ad-hoc solution on that account to emphase the effect. In a real head tracking stereoscopic system you do slightly different. Technically headZ should be either used to calculated the near plane distance or be derived from it.

Anyway the main ideas is, that the head is located at some distance from the projection plane, and the center point is shifted in relative units of the projection. So you must scale relative headX, headY with the actual head distance to the projection plane to make the apex correction work.

Update due to comment/request

So far we've looked at only one dimension when converting field of view (fov) to screen span. For the image to be undistorted the aspect ratio of the [left, right] / [bottom, top] extents of the near clipping plane must match the aspect ratio of the viewport width/height.

If we choose to define the FoV angle to be the vertical FoV, then the horizontal size of the near clipping plane extents is the size of the vertical near clipping plane extents scaled with the with/height aspect ratio.

This is nothing special about off-axis projection, but can be found in every perspective projection helper function; compare the source code of gluPerspective for reference:

void GLAPIENTRY
gluPerspective(GLdouble fovy, GLdouble aspect, GLdouble zNear, GLdouble zFar)
{
   GLdouble xmin, xmax, ymin, ymax;

   ymax = zNear * tan(fovy * M_PI / 360.0); // M_PI / 360.0 == DEG_TO_RAD
   ymin = -ymax;

   xmin = ymin * aspect;
   xmax = ymax * aspect;

   glFrustum(xmin, xmax, ymin, ymax, zNear, zFar);
}

And if we consider the near clipping plane extents to be [-aspect, aspect]×[-1, 1] then of course the headX position is not in the normalized range [-1, 1] but must be given in the range [-aspect, aspect] as well.

If you look at the paper you linked, you'll find that for each screen the head position as reported by the tracker is transformed in absolute coordinates relative to the screen.

---

Two weeks ago I had the opportunity to test a display system called "Z space" where a polarized stereo display had been combined with a head tracker creating an off-axis frustum / look-at combination that matched your physical head position in front of the display. It also offers a "pen" to interact with the 3D scene in front of you. This is one of the most impressive things I've seen in the last few years and I'm currently begging my boss to buy us one :)