Tangent, binormal and other shader-related things

Question

Many of the mapping techniques including normal bump mapping, parallax mapping and others require the special per-vertex tangent-space basis (tangent, normal, binormal / bitangent).

This obviously means that my models should not only export vertex positions, texture coordinates and approximated per-vertex normals, but also one of the tangent space basis vectors (usually tangent), because the other one can be found directly in the shader using cross(tangent, normal).

Note that position, normal, uv and tangents actually depend on each other the following way (you have to know everything else about the vertex to prepare the tangent basis).

position -> normal ->  tangents
            uv     ->

Now - how is this sort of things handled in modern 3D games / rendering engines?

Do they actually supply the normals, tangents and the uv coordinates per each vertex or can they somehow be calculated in runtime? Should they be the part of the model data or should they be a runtime-only property?

I also know that when using Direct3D10+ using the geometry shaders one can actually prepare the normals and the the tangents right in runtime (obviously, because we have access to the vertices in each triangle) - is it worth it or should these things be always precomputed?

Answer 1

My view is as follows:

UV coordinates: clearly precomputed. UV unwrapping of a complex model is not a trivial, algorithmic task and usually involves manual placing of seams for best result. Also, if your model comes with a texture skin, then the UVs need to match the skin, so in this case they may not be computed during runtime.

Normals: theoretically you could precompute them in a geometry shader or even on CPU during loading. But there's one thing: the real models in games are often NOT the original models, but versions with lower polygon count. In the final model used in-game, you have less vertices, but the per-vertex normals are calcuated more accurately from the original high-poly model (which is also used for ambient occlusion calculation, etc). Therefore, having the normals precalculated is also feasible.

Also, in many games a texture with per-texel normal map (usually in tangent space, AFAIK) is provided together with the color texture for each model, for the purposes of parallel mapping later in the pixel shader.

Therefore it looks like my suggestion is clear - go for precomputed data, have better details and save on either shader processing or loading time - unless you need to worry about the model size (i.e. GPU memory limit) - only then having normals calculated in geometry shader would save you some space.

Answer 2

Generally you'll want to precompute as much as possible. Most engines will have a custom written tool that can take in raw data from 3D software, and massage it into a format for run time (adding in tangents and bitangents, etc). However...

In certain situations the per-vertex memory overhead can be a particular performance concern. This is because there's an overhead associated with moving data about the system -- it's "hidden" because it's not an explicit as an instruction added into a shader, so it's a little more advanced.

In these cases, the bitangent may not be precomputed, and instead be constructed at runtime (in order to reduce the size of the vertex in memory). Bitangents are appropriate for this because they can be built from a normal, tangent and a directionality flag -- which are all properties of a single vertex.

Normals, some UV mappings and other things require knowledge of the "topology" of the mesh -- and this topology isn't usually present in runtime data. They also require extra markings from the artists; like sharp edges and textures seams, etc. For this reason, normals, UVs and the first tangent will almost always be precalculated for all but very specialised types of geometry.

Remember that that the representation of a model in Max, Maya or Blender is very different than the run time version.

Editor software

• soup of triangles and quads, with neighbour information
• edges marked as sharp/soft
• a single vertex can have multiple normals and multiple UV coords

Run time (for DirectX, OpenGL, etc)

• cache friendly collection of triangle strips and triangle fans
• edges are implicit, there is no extra information tied to them
• each vertex has only one position, one normal and one UV coord (per texture)
• sometimes exists at multiple levels of detail

In general, you lose a lot of information that cannot be reconstructed. Your artists will be happiest if you do as much as you can with the original Max/Maya version of the geometry.