Omnidirectional shadow mapping with depth cubemap

Question

I'm working with omnidirectional point lights. I already implemented shadow mapping using a cubemap texture as color attachement of 6 framebuffers, and encoding the light-to-fragment distance in each pixel of it.

Now I would like, if this is possible, to change my implementation this way:

• 1) attach a depth cubemap texture to the depth buffer of my framebuffers, instead of colors.
• 2) render depth only, do not write color in this pass
• 3) in the main pass, read the depth from the cubemap texture, convert it to a distance, and check whether the current fragment is occluded by the light or not.

My problem comes when converting back a depth value from the cubemap into a distance. I use the light-to-fragment vector (in world space) to fetch my depth value in the cubemap. At this point, I don't know which of the six faces is being used, nor what 2D texture coordinates match the depth value I'm reading. Then how can I convert that depth value to a distance?

Here are snippets of my code to illustrate:

Depth texture:

glGenTextures(1, &TextureHandle);
glBindTexture(GL_TEXTURE_CUBE_MAP, TextureHandle);
for (int i = 0; i < 6; ++i)
    glTexImage2D(GL_TEXTURE_CUBE_MAP_POSITIVE_X + i, 0, GL_DEPTH_COMPONENT,
              Width, Height, 0, GL_DEPTH_COMPONENT, GL_FLOAT, 0);

glTexParameteri(GL_TEXTURE_CUBE_MAP, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_CUBE_MAP, GL_TEXTURE_MIN_FILTER, GL_NEAREST);

glTexParameteri(GL_TEXTURE_CUBE_MAP, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_CUBE_MAP, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

Framebuffers construction:

for (int i = 0; i < 6; ++i)
{
    glGenFramebuffers(1, &FBO->FrameBufferID);
    glBindFramebuffer(GL_FRAMEBUFFER, FBO->FrameBufferID);
    glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT,
            GL_TEXTURE_CUBE_MAP_POSITIVE_X + i, TextureHandle, 0);
    glDrawBuffer(GL_NONE);
}

The piece of fragment shader I'm trying to write to achieve my code:

float ComputeShadowFactor(samplerCubeShadow ShadowCubeMap, vec3 VertToLightWS)
{   
    float ShadowVec = texture(ShadowCubeMap, vec4(VertToLightWS, 1.0));
    ShadowVec = DepthValueToDistance(ShadowVec);
    if (ShadowVec * ShadowVec > dot(VertToLightWS, VertToLightWS))
        return 1.0;

    return 0.0;
}

The DepthValueToDistance function being my actual problem.

Answer 1

So, the solution was to convert the light-to-fragment vector to a depth value, instead of converting the depth read from the cubemap into a distance.

Here is the modified shader code:

float VectorToDepthValue(vec3 Vec)
{
    vec3 AbsVec = abs(Vec);
    float LocalZcomp = max(AbsVec.x, max(AbsVec.y, AbsVec.z));

    const float f = 2048.0;
    const float n = 1.0;
    float NormZComp = (f+n) / (f-n) - (2*f*n)/(f-n)/LocalZcomp;
    return (NormZComp + 1.0) * 0.5;
}

float ComputeShadowFactor(samplerCubeShadow ShadowCubeMap, vec3 VertToLightWS)
{   
    float ShadowVec = texture(ShadowCubeMap, vec4(VertToLightWS, 1.0));
    if (ShadowVec + 0.0001 > VectorToDepthValue(VertToLightWS))
        return 1.0;

    return 0.0;
}

Explaination on VectorToDepthValue(vec3 Vec) :

LocalZComp corresponds to what would be the Z-component of the given Vec into the matching frustum of the cubemap. It's actually the largest component of Vec (for instance if Vec.y is the biggest component, we will look either on the Y+ or the Y- face of the cubemap).

If you look at this wikipedia article, you will understand the math just after (I kept it in a formal form for understanding), which simply convert the LocalZComp into a normalized Z value (between in [-1..1]) and then map it into [0..1] which is the actual range for depth buffer values. (assuming you didn't change it). n and f are the near and far values of the frustums used to generate the cubemap.

ComputeShadowFactor then just compare the depth value from the cubemap with the depth value computed from the fragment-to-light vector (named VertToLightWS here), also add a small depth bias (which was missing in the question), and returns 1 if the fragment is not occluded by the light.