androidopengl-esglslandroid-mediacodec

Is the value read from the GL_TEXTURE_EXTERNAL_OES texture linear rgb?

I use MediaCodec to decode h264 stream from the server, and render frames. The server encoded frames which is sRGB. The decoder stores the result into oes texture. I know that samplerExternalOES will convert YUV to RGB.

There is fragment shader code:

#extension GL_OES_EGL_image_external_essl3 : enable
#extension GL_OES_EGL_image_external : enable
in lowp vec2 uv;
out lowp vec4 FragColor;
uniform samplerExternalOES OesTexture;
void main()
{
    vec3 tmpColor = texture(OesTexture, uv).rgb;
    ...
    FragColor= vec4(tmpColor, 1.0);
}

Is "tmpColor" linear rgb? Or sRGB?

Solution

  • If the color of the video frame used as an external texture is non-linear, then the sampled color is also going to be non-linear.

    According to the OES_EGL_image_external extension:

    Sampling an external texture will return an RGBA vector in the same colorspace as the source image. If the source image is stored in YUV (or some other basis) then the YUV values will be transformed to RGB values. (But these RGB values will be in the same colorspace as the original image. Colorspace here includes the linear or non-linear encoding of the samples. For example, if the original image is in the sRGB color space then the RGB value returned by the sampler will also be sRGB, and if the original image is stored in ITU-R Rec. 601 YV12 then the RGB value returned by the sampler will be an RGB value in the ITU-R Rec. 601 colorspace.)

    For regular images and textures, there are other extensions (EXT_sRGB, EXT_texture_sRGB) that enable automatic gamma encoding / decoding.

    But video is more complex. It can be encoded using various color standards (BT.601, BT.709, BT.2020) and transfer functions (linear, gamma, HLG, PQ). These properties are specified in the media track format.

    See: