transform world space normals to screen space normals

Question

What is the proper way to transform surface normals defined in world space to normals in screen space? I don't think they can simply be multiplied by the projection matrix, because perspective division transforms things into clip space and as far as I understand, in this space planes that are coplanar to the image plane remain coplanar.

But the transformation I'm looking for should result in transformed normals such that the blue world normals depicted in graphic A should result in differing screenspace normals (because, even though their planes are coplanar to the image plane, they do not face the camera) - on the other hand the depicted normals in graphic B should after the transformation be (more or less) equal, since their surfaces face the camera.

What transformation am I looking for? how to calculate it?

I need this for some screen-space effects.

Answer 1

You're looking for Transformation Matrix with Respect to a Basis. There is quite fine video about steps how to calculate this matrix. Check KhanAcademy video.

EDIT

You didn't provide any information about technology which are you using (OGL, DX, shaders, own projection, ...), but you should definitely read some articles about tangent space, e.g. Messing with Tangent Space..

Answer 2

Try to do it as usually done when transforming normals from model(to view) space - using inverse-transpose matrix.

Answer 3

Supposing you want to transform the normal N at point P by an arbitrary 4x4 matrix. Let's represent the matrix with the model part T (having rotation, scale, and translation) and the part with the last row as W. This allows us to say that:

Transform(P) = (T * P) / dot(W,P)

Now, the projected normal can be seen as a limit(d -> 0) of:

(Transform(P+dN) - Transform(P)) / d

Expanding this equation and then crossing out "d" from both nominator and denominator leads to this formula:

dot(W,N) * (Transform(N) - Transform(P)) / dot(W,P)

This is untested and unverified, but I thought given the lack of positive answers to this question I'd give it a shot ;)