Read and display animations from *.FBX file

Question

I want to read 3d model from fbx file and display it within openGL es 2.0 engine on iPhone (not using Unity) and also I want to display animations for read 3d object.

How can i get animations from fbx file?

Currently I able to get list of poses names with, as I understood, coming with transformation matrix, full list of poses in layer, stacks, layers, and a lot of curves.

How this all information can be combined for displaying proper animations?

I also try to parse some info within TakeInfo, but result a little bit strange as for me, for example:

    FbxTakeInfo *ltakeInfo  =  pScene->GetTakeInfo(lAnimStack->GetName());
    FbxTime start = ltakeInfo->mLocalTimeSpan.GetStart();
    FbxTime end = ltakeInfo->mLocalTimeSpan.GetStop();

    self.startTime = start.GetSecondDouble();
    self.endTime = end.GetSecondDouble();

here i got start = 0 and end = 0.014 for each of parsed layers, so i guees this is incorrect (fbx file that I want to display contains 1 mesh with simple 5 sec duration animations).

---

Update

After few hours of investigating I got next things:

For reference this is structure of test obj that i want to display:

Here u can see a lot of bones (more concrete - 19) I able to get (as noted above) a 19 animated obj in list (like names of bone/obj) and 19 groups of curves within 151 frames each (with frame rate 30 exactly 5 sec of anim. - 30*5=150 + 1 last identity matrix).

If I try use each of curve group to my mesh one by one (I able to parse 1 mesh only) I see animation of different part of mesh applied to all mesh (for example vertical rotation or horizontal translation), so I think that this curves in each group should be applied exactly to specific bone and as result i will got animation for my mesh. The problem is that i don't know how to animate only selected bone vertex part only.

Now the problem is how to apply this all animations divided in to separate group specific to bone to whole obj (because I have only one mesh)?

How can I get 1 global curve's list for each frame from list with all curve's groups?

---

Update2

Thanks to @codetiger advice I follow instruction within provided link in comment and with that technique I'm able to retrieve list of bone-specific mat's with start and end time and required transforms, but this is almost same like I got before with curves - the only difference that with curves I should create mat from 9 curves (translate/ scale/ rotate for x y z) instead of using full matrix, but problem are still present - how can i combine them in 1 global matrix?

The code that I use (found few links for it):

FbxNode* modelNode = _fbxScene->GetRootNode();
FbxAMatrix geometryTransform = GetGeometryTransformation(modelNode);
for (unsigned int deformerIndex = 0; deformerIndex < numOfDeformers; ++deformerIndex) {
    FbxSkin* currSkin = reinterpret_cast<FbxSkin*>(mesh->GetDeformer(deformerIndex, FbxDeformer::eSkin));
    if (!currSkin) {
        continue;
    }
    unsigned int numOfClusters = currSkin->GetClusterCount();
    for (unsigned int clusterIndex = 0; clusterIndex < numOfClusters; ++clusterIndex) {
        FbxCluster* currCluster = currSkin->GetCluster(clusterIndex);
        std::string currJointName = currCluster->GetLink()->GetName();

        FbxAMatrix transformMatrix;
        FbxAMatrix transformLinkMatrix;
        FbxAMatrix globalBindposeInverseMatrix;

        currCluster->GetTransformMatrix(transformMatrix);   
        currCluster->GetTransformLinkMatrix(transformLinkMatrix);   
        globalBindposeInverseMatrix = transformLinkMatrix.Inverse() * transformMatrix * geometryTransform;

        FbxAnimStack* currAnimStack = _fbxScene->GetSrcObject<FbxAnimStack>(0);
        FbxString animStackName = currAnimStack->GetName();
        char *mAnimationName = animStackName.Buffer();
        FbxTakeInfo* takeInfo = _fbxScene->GetTakeInfo(animStackName);
        FbxTime start = takeInfo->mLocalTimeSpan.GetStart();
        FbxTime end = takeInfo->mLocalTimeSpan.GetStop();
        FbxLongLong mAnimationLength = end.GetFrameCount(FbxTime::eFrames24) - start.GetFrameCount(FbxTime::eFrames24) + 1;

        for (FbxLongLong i = start.GetFrameCount(FbxTime::eFrames24); i <= end.GetFrameCount(FbxTime::eFrames24); ++i) {
            FbxTime currTime;
            currTime.SetFrame(i, FbxTime::eFrames24);

            FbxAMatrix currentTransformOffset = modelNode->EvaluateGlobalTransform(currTime) * geometryTransform;
            FbxAMatrix mat = currentTransformOffset.Inverse() * currCluster->GetLink()->EvaluateGlobalTransform(currTime);

        }
    }
}

Here I receive 121 matrix instead of 151 - but duration of some matrix transforms take more than duration of 1 frame draw, so the q-ty here I think also correct

float duration = end.GetSecondDouble() - start.GetSecondDouble(); //return 5 as and expected

I guess that that Autodesk SDK there is a way to get 1 Global transform per each drawCall

Any suggestions? Is this possible?

---

Add bounty for anyone who can describe how to display animations on iPhone within openGLES 2.0 with Autodesk SDK... (sorry typo instead of Facebook)

---

Here what i'm able to get

Original object in blender

If i draw bone separately (same VBO with different transform and only indices for each bone accordantly)

Answer 1

Here is how to render animated mesh in OpenGL ES. This will give you fare information on what details you need to read from the file.

Method 1: (GPU Skinning)

This method works only on limited number of bones based on the capabilities of the hardware. Usually it depends on the amount of matrices you can send to the shader.

Bind the mesh information to the GPU using BindAttributes once and send the matrixes to the shader in uniforms.

Step 1: Read all Bone Matrix and create an array of Matrices and send this data to the shader in uniforms.

Step 2: In the Vertex Shader, calculate the gl_position like the below shader

attribute vec3 vertPosition;
attribute vec3 vertNormal;
attribute vec2 texCoord1;
attribute vec3 vertTangent;
attribute vec3 vertBitangent;
attribute vec4 optionalData1;
attribute vec4 optionalData2;

uniform mat4 mvp, jointTransforms[jointsSize];

void main()
{
    mat4 finalMatrix;

    int jointId = int(optionalData1.x);
    if(jointId > 0)
        finalMatrix = jointTransforms[jointId - 1] * optionalData2.x;

    jointId = int(optionalData1.y);
    if(jointId > 0)
        finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.y);

    jointId = int( optionalData1.z);
    if(jointId > 0)
        finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.z);

    jointId = int( optionalData1.w);
    if(jointId > 0)
        finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.w);

    gl_Position = mvp * finalMatrix * vec4(vertPosition, 1.0);
}

In this shader, I've send the weight paint information in the attributes optionalData1 & optionalData2. The data is packed like this: (BoneId1, BoneId2, BoneId3, BoneId4) and (Weight4Bone1, Weight4Bone2, Weight4Bone3, Weight4Bone4). So in this case you are limited with a maximum of 4 bones affecting each attributes. The fragment shader part is usual.

Method 2: (CPU Skinning)

If you cannot live with the limitation in the GPU skinning, then the only way is to do the calculations in the CPU side.

Step 1: Calculate the vertex position in the current frame by multiplying the matrixes that belong to the bones that affects the vertex.

Step 2: Collect the new positions for the current frame and send the information to GPU in Vertex Attributes.

Step 3: Recalculate new Vertex positions in every frame and update the attributes buffer.

This method bring the load of the calculations to the CPU.