What is glSampleMask?

Question

The OpenGL wiki that says that glSampleMask() sets the value of a sub-word of the sample mask. (which does not make any sense to me.)

So what does sample masking mean? (I am aware of glColorMask, but can not co-relate it to glSampleMask). I am assuming it has something to do with what samples are finally chosen in computing the final color? But suppose I have 4 samples per pixel and I set glSampleMask(0, 0xFF), what exactly happens?

What does mask number (0 in this case) denote?

Is gl_SamplerMask the shader equivalent of glSampleMask?

Can somebody explain the terms here?

Answer 1

As answer was obscure, it now almost completely rewritten.

Spec says:

if SAMPLE_MASK is enabled, the fragment coverage is ANDed with the
coverage value SAMPLE_MASK_VALUE

Take a look at image at http://msdn.microsoft.com/en-us/library/windows/desktop/cc627092%28v=vs.85%29.aspx#Multisample

Dots on this image shows sample locations. Sample mask allows you to specify bitfield of which samples will be written. If you specified e.g. 0x1, only first sample will be written, 0x2 - second, 0x3 - first and second, and so on. Shader is evaluated only once and samples values are inter/extra-polated, so for each shader execution you'll have 4, 8, 16, etc. samples (depending on how strong MSAA you've enabled).

How this could be used in practice? E.g. you could write one draw to one sample, one into another, etc., and then read them all in shader and combine in your preferred way. For instance

// ... attach FBO
glEnable(GL_SAMPLE_MASK);
glSampleMaski(0, 0x1);
// ... draw scene0
glSampleMaski(0, 0x2);
// ... draw scene1

glDisable(GL_SAMPLE_MASK);
// ... bind FBO to texture
// ... bind tricky shader
// ... draw it, reading both results from different samples and combining them

You could read different samples from FBO with texelFetch(texture, texcoords, i), where i is number of sample, starting with zero.

OpenGL super bible, 5th edition, contains example on using this technique for order-independent (although quite limited) transparency.

gl_SampleMask array in shader allows you perform the same thing, but on per-fragment basis, so different fragments of same object could use different sample masks - and be written into different samples as a results.