Can someone please explain this Fragment Shader? It is a Chroma Key Filter (Green screen effect)

Question

I'm trying to understand how this chroma key filter works. Chroma Key, if you don't know, is a green screen effect. Would someone be able to explain how some of these functions work and what they are doing exactly?

float maskY = 0.2989 * colorToReplace.r + 0.5866 * colorToReplace.g + 0.1145 * colorToReplace.b;
 float maskCr = 0.7132 * (colorToReplace.r - maskY);
 float maskCb = 0.5647 * (colorToReplace.b - maskY);

 float Y = 0.2989 * textureColor.r + 0.5866 * textureColor.g + 0.1145 * textureColor.b;
 float Cr = 0.7132 * (textureColor.r - Y);
 float Cb = 0.5647 * (textureColor.b - Y);

 float blendValue = smoothstep(thresholdSensitivity, thresholdSensitivity + smoothing, distance(vec2(Cr, Cb), vec2(maskCr, maskCb)));


 gl_FragColor = vec4(textureColor.rgb * blendValue, 1.0 * blendValue);

I understand the first 6 lines (converting the color to replace, which is probably green, and the texture color to the YCrCb color system).

This fragment shader has two input float values: thresholdSensitivity and Smoothing.

• Threshold Sensitivity controls how similar pixels need to be colored to be replaced.
• Smoothing controls how gradually similar colors are replaced in the image.

I don't understand how those values are used in the blendValue line. What does blendValue compute? How does the blendValue line and the gl_FragColor line actually create the green screen effect?

Answer 1

The smoothstep function in GLSL evaluates a smooth cubic curve over an interval (specified by the first two parameters). As compared to GLSL's mix function, which linearly blends its parameters as:

smoothstep uses a Hermite cubic polynomial to determine the value

In your shader, blendValue is a smooth interpolation of your smoothing value based on the distance between the red and blue chrominance values.

Finally, gl_FragColor specifies the final fragment color (before blending, which occurs after completion of the fragment shader). In your case, it's the modulated value read from the input image, and a modulated alpha value for translucency.