Use RGB texture as alpha values/Subpixel font rendering in OpenGL

Question

Currently I'm using FreeType in subpixel mode and take the largest color of each pixel as alpha value, with the following fragment shader:

    uniform sampler2D Image;
    uniform vec4 Color;

    smooth in vec2 vVaryingTexCoord;

    out vec4 vFragColor;

    void main(void){
        vec4 color = texture(Image, vVaryingTexCoord);
        vFragColor = color * Color;
    }

This works fine for dark backgrounds, but on lighter ones the border pixels show (e.g. when a text pixel is (1,0,0)). To make it work with brighter backgrounds, I'd need to pass the background color and do the blending myself, which starts breaking down once I move to more complex backgrounds.

Is there a way to use the RGB values from FreeType as alpha values for a solid color (which is passed to the shader)? This formula basically, where b = background pixel, t = current text pixel, c = static color:

b*((1,1,1) - t) + t*c.rgb*c.a

I think drawing everything else first and passing that framebuffer to the font shader would work, but that seems a bit overkill. Is there a way doing this in the OpenGL blend stage? I tried playing around with glBlendFunc and such, but didn't get anywhere.

Answer 1

It's possible using Dual Source Blending, available since OpenGL 3.3. This spec draft even mentions subpixel rendering as use case. All that is needed to make it work:

glBlendFunc(GL_SRC1_COLOR, GL_ONE_MINUS_SRC1_COLOR);

(don't forget to enable GL_BLEND, it happens to me all the time :D)

Specify dual output in the fragment shader: (You can bind by name instead if you want, see spec)

layout(location = 0, index = 0) out vec4 color;
layout(location = 0, index = 1) out vec4 colorMask;

In main:

color = StaticColor;
colorMask = StaticColor.a*texel;

Where StaticColor is the global text color uniform, and texel is the current pixel value of the glyph.