Currently I'm using FreeType in subpixel mode and take the largest color of each pixel as alpha value, with the following fragment shader:
uniform sampler2D Image;
uniform vec4 Color;
smooth in vec2 vVaryingTexCoord;
out vec4 vFragColor;
void main(void){
vec4 color = texture(Image, vVaryingTexCoord);
vFragColor = color * Color;
}
This works fine for dark backgrounds, but on lighter ones the border pixels show (e.g. when a text pixel is (1,0,0)). To make it work with brighter backgrounds, I'd need to pass the background color and do the blending myself, which starts breaking down once I move to more complex backgrounds.
Is there a way to use the RGB values from FreeType as alpha values for a solid color (which is passed to the shader)? This formula basically, where b = background pixel, t = current text pixel, c = static color:
b*((1,1,1) - t) + t*c.rgb*c.a
I think drawing everything else first and passing that framebuffer to the font shader would work, but that seems a bit overkill. Is there a way doing this in the OpenGL blend stage? I tried playing around with glBlendFunc and such, but didn't get anywhere.
It's possible using Dual Source Blending, available since OpenGL 3.3. This spec draft even mentions subpixel rendering as use case. All that is needed to make it work:
glBlendFunc(GL_SRC1_COLOR, GL_ONE_MINUS_SRC1_COLOR);
(don't forget to enable GL_BLEND, it happens to me all the time :D)
Specify dual output in the fragment shader: (You can bind by name instead if you want, see spec)
layout(location = 0, index = 0) out vec4 color;
layout(location = 0, index = 1) out vec4 colorMask;
In main:
color = StaticColor;
colorMask = StaticColor.a*texel;
Where StaticColor is the global text color uniform, and texel is the current pixel value of the glyph.