Importing 3D skeletal animations from .smd model

Question

I'm writing an .smd importer, and i'm stuck at the skeletal animation part. The problem is that i dont know exactly how it works. I'm using this this to write the exporter, but it doesn't show how to use the information stored in file.

I imagine that all vertexes with the same bone id should be grouped, translated and rotated, because you cant rotate each vertex. But i dont know if i'm right, and even if i were, i still don't know how to do that by scripting...

So the question is: how do i use the skeletal animation info stored in the file?

Answer 1

I'm not familiar with the SMD format specifically, but here goes...

Note: This answer assumes you know how to construct a composite transform for an object/node. This is the matrix that combines its translation, rotation and scale (although scale is not used in SMD, it seems). Also, matrix multiplication, matrix inversion and matrix * vector multiplication are used.

Bones and animation

A model's bone nodes form a tree; each bone has a parent bone, except for the root bone (the nodes section). Each node has its own, local transform (position and rotation).

Local node transform: A node's local transform is a 4x4 matrix constructed from its position and rotation, which transforms points from its local space to its parent node's space: If a vector representing a position in the node's space, multiplying it with the matrix results in that vector in the parent's space. See this link, for details on how to do this. Google a bit for more.

In SMD, bone transforms are defined only in keyframes in animations (the skeleton section). A "reference" SMD file has a single frame of animation; a position and rotation for each bone node in the reference position of the model.

An animation SMD file has an animation sequence with multiple frames, each specifying different transforms for (some) bones. When playing an animation, you interpolate between frames based on their times and the current scene/game time and come up with a transform (position + rotation) for each bone.

Getting the at-rest bone transforms

In pre-processing (when loading the mesh or so), you need to calculate the so-called "at-rest" bone transforms. These are the model-to-bone space transforms for each bone, when it's in the reference position. Here's why:

All vertex positions are defined in model space, but ultimately, vertex transformation must be done starting from bone space, because you want vertices to move with individual bones. Therefore, vertex positions must first be transformed to bone space. This is where the at-rest bone transforms come in.

So the at-rest transform we're looking for transforms a vertex from model space to bone space. Put all bones in the reference position. Walk the tree starting at the root node, and concatenate transform matrices. So for example, for an upper arm node you'd get the transform:

transform = root * spine * shoulderR * upperArmR

However, this is the transform from the space of the upper arm to model space. So simply invert the matrix to get the at-rest bone transform. Do this for every bone and store these matrices.

Note that the at-rest transforms do not change over time; they're fixed based on the model's reference position.

Vertex/bone associations

Each vertex is associated with 1 or more bone nodes. For each such association, the vertex has a corresponding weight. Usually, all weights sum to 1. In SMD, these associations are defined in the triangles segment. Based on the page you linked to, the format is:

triangles
my_material
bone_id    x y z    nx ny nz    u v    bone_links

This defines a vertex at (x, y, z) and intially associates it with bone bone_id (with a weight of 1 I assume). The bone_links section can (sort of) override this and specify multiple associations as follows:

bone_links = num_links bone_id[0] weight[0] bone_id[1] weight[1] ... etc.

If the weights do not add up to 1, remaining weight goes to the association with the original bone_id.

So an example vertex associated with bones 0, 1 and 2 would be:

0    x y z   nx ny nz   u v    3   0 0.15  1 0.35  2 0.5

Vertex transformation

Here's where we finally determine the position of vertices based on current bone transforms. As stated earlier; based on the current time and animation, you determine (interpolate) the current bone transforms. For each bone, calculate the bone-to-world transform. Example (we saw this earlier):

boneToWorld = root * spine * shoulderR * upperArmR

Now, for a vertex associated to a single bone, the following gives its animated/skinned position:

vertexPosAnimated = boneToWorld * boneAtRest * vertexPosModel

This first transforms the vertex position from model space (vertexPosModel) to the space of the bone it's associated with (this transform doesn't change over time). Then, using the current position of the bone, the vertex is transformed from bone to model space again. This allows it to move with the bone as its transform changes.

Observe that, when the bone is currently in its at-rest position, boneAtRest is the inverse of boneToWorld, so boneToWorld * boneAtRest is the identity matrix, so the vertex position remains unchanged, which is correct!

Finally, since vertices can be associated with multiple bones, instead of the above we calculate a weighted sum of the above for each associated bone. For example, for a vertex associated with 3 bones:

vertexPosAnimated =
    boneToWorld[0] * boneAtRest[0] * vertexPosModel * weight[0] +
    boneToWorld[1] * boneAtRest[1] * vertexPosModel * weight[1] +
    boneToWorld[2] * boneAtRest[2] * vertexPosModel * weight[2];

Final thoughts

These are some broad strokes and I haven't even discussed shader implementation (if that's what you're going to do), but I think I've covered all principles, and it's already a pretty long answer.

One thing I've found helpful in the past for 3D engine development is the M3G documentation (the old Java Mobile 3D API). The entry on SkinnedMesh basically describes what I posted here as well.

Also, try and understand the concept of transforming a vertex from model to bone space using the at-rest bone transform, and back again using its current transform. That's the key to the whole thing.

Good luck!