How to get CATransform3D from Projection and ModelView matrices?

Question

all,

I've got an iphone project that draws a 3D model using OpenGL-ES for a given model view matrix and given projection matrix. I needed to replace 3D model with CALayer, so I put values of model view matrix into CATransform3D structure and assigned it to layer.transform. It worked well, layer was visible and moved on the screen as expected, but after some time I realized that my layers behavior is not precise enough and I should take projection matrix into account. And then a problem appeared: when I simply concatenate two matrices my layer looks odd (it is very small, about 2 pixels, while it is supposed to be about 300, as it is far far away) or is not visible at all. How can I solve it?

Here is the piece of code:

- (void)adjustImageObjectWithUserInfo:(NSDictionary *)userInfo
{
    NSNumber *objectID = [userInfo objectForKey:kObjectIDKey];
    CALayer *layer = [self.imageLayers objectForKey:objectID];
    if (!layer) { return; }

    CATransform3D transform = CATransform3DIdentity;
    NSArray *modelViewMatrix = [userInfo objectForKey:kModelViewMatrixKey];

      // Get raw model view matrix;
    CGFloat *p = (CGFloat *)&transform;
    for (int i = 0; i < 16; ++i)
    {
        *p = [[modelViewMatrix objectAtIndex:i] floatValue];
        ++p;
    }

      // Rotate around +z for Pi/2 
    transform = CATransform3DConcat(transform, CATransform3DMakeRotation(M_PI_2, 0, 0, 1));

      // Project with projection matrix
    transform = CATransform3DConcat(transform, _projectionMatrix);

    layer.transform = transform; 
}

Any help will be appreciated.

Answer 1

I recently ran into this exact problem (I was using ARToolKit as well) and I was very disappointed to see that you hadn't figured out the answer. I imagine you have moved on now but I figured it out and I am posting it for any other lost soul who might come through with this same problem.

The most confusing thing for me was that everyone talks about making a CALayer perspective transform by setting the m34 variable to a small negative number. Although that does work it is not very informative at all. What I eventually realized is that the transform works exactly like every other transform, it is a column major transformation matrix for homogenous coordinates. The only special case is that it must combine the model view and projection matrices, and then scale to the size of the openGL viewport all in one matrix. I started by trying to use a matrix in the style where m34 is a small negative number as explained in much greater detail here but eventually switched to open GL style perspective transforms as explained here. They are in fact equivalent to one another they just represent different ways of thinking about the transform.

In our case we are trying to make the CALayer transform exactly replicate an open GL transform. All that requires is multiplying together the modelview,projection,and scaling matrices and flipping the y axis to account for the fact that the device screen origin is top left and open GL is bottom left. As long as the layer anchor is at (.5,.5) and its position is exactly in the center of the screen the result will be identical to the open GL's transform

void attach_CALayer_to_marker(CATransform3D* transform, Matrix4 modelView, Matrix4 openGL_projection, Vector2 GLViewportSize)
{
//Important: This function assumes that the CA layer has its origin in the 
//exact center of the screen.

Matrix4 flipY = {   1, 0,0,0,
                    0,-1,0,0,
                    0, 0,1,0,
                    0, 0,0,1}; 

//instead of -1 to 1 we want our result to go from -width/2 to width/2, same 
//for height
CGFloat ScreenScale = [[UIScreen mainScreen] scale];
float xscl = GLViewportSize.x/ScreenScale/2;
float yscl = GLViewportSize.y/ScreenScale/2;

//The open GL perspective matrix projects onto a 2x2x2 cube.  To get it onto the
    //device screen it needs to be scaled to the correct size but
//maintaining the aspect ratio specified by the open GL window.
    Matrix4 scalingMatrix = {xscl,0   ,0,0,
                               0,   yscl,0,0,
                               0,   0   ,1,0,
                               0,   0   ,0,1};

//Open GL measures y from the bottom and CALayers measure from the top so at the
//end the entire projection must be flipped over the xz plane.
//When that happens the contents of the CALayer will get flipped upside down.
//To correct for that they are flipped updside down at the very beginning,
//they will then be flipped right side up at the end.

Matrix flipped = MatrixMakeFromProduct(modelView, flipY);
Matrix unscaled = MatrixMakeFromProduct(openGL_projection, flipped);
Matrix scaled = MatrixMakeFromProduct(scalingMatrix, unscaled);

//flip over xz plane to move origin to bottom instead of top
Matrix Final = SiMatrix4MakeFromProduct(flipY, scaled);
*transform = convert_your_matrix_object_to_CATransform3D(Final);
}

This function takes the open GLprojection, and openGL view size and uses them to generate the correct transform for the CALayer. The CALayer size should be specified in the units of the open GL scene. The OpenGL viewport actually contains 4 variables, [xoffset,yoffset,x,y] but the first two are not relevant because the Origin of the CALayer is put in the center of the screen to correspond to the OpenGL 3d Origin.

Just replace Matrix with whatever generic 4x4 column major matrix class you have access to. Anything will work just make sure you multiply your matrices in the right order. All this is essentially doing is replicating the OpenGL pipeline (minus clipping).