c++graphicsglslvulkannormals

Tangent seams in Vulkan using GLSL and C++


I am writing a small Vulkan toy renderer using tinygltf to load my models. For the past two weeks I've been trying to implement normal mapping into the renderer, but I've been running into issues.

I think I narrowed it down to my tangent / TBN matrix calculations, but at this point it could be anything. I retrieved all my models from the sample models on glTF editor (cannot link due to reputation being to low). As you can see in the attached images, when using the sampled and TBN transformed normal, there is a big seam running through the middle of my model. To make sure this isn't an issue with the model itself, I tried the following:

  1. I imported the model into Blender to see if the same seam is visible, which isn't the case. Everything looks fine in Blender.
  2. I tried out a few different models (Sponza, Damaged Helmet, Corset), but they all had the same issue, weird tangent seams everywhere.

Some models didn't have tangents so I went into Blender and exported the models with tangents, but the seam was still there.

I then tried calculating the tangents and bitangents myself when importing the models, using the explanation provided here (page 5, Listing 7.4). But even this didn't fix my issues, the imported and calculated tangents were exactly the same.

Things I have tried:

  1. Transform the light and view direction from world to tangent space inside the vertex shader as seen in the Learn OpenGL article but to no avail.

Vertex shader:

void main() 
{
    vec4 worldPos = primitive.model * vec4(inPosition, 1.0f);
    gl_Position = ubo.viewProjection * worldPos;

    mat3 modelM3 = transpose(inverse(mat3(primitive.model)));
    vec3 T = normalize(modelM3 * inTangent.xyz);
    vec3 N = normalize(modelM3 * inNormal);
    vec3 B = normalize(cross(N, T)) * inTangent.w;

    // Transposed matrix to transform light and view vectors from world to tangent space.
    mat3 TBN = mat3(
    T.x, B.x, N.x,
    T.y, B.y, N.y,
    T.z, B.z, N.z
    );

    vs_out.normal = N;
    vs_out.tangent = vec4(T, inTangent.w);
    vs_out.bitangent = normalize(modelM3 * inBitangent);
    vs_out.texcoord = inTexcoord;

    vs_out.lightDir = ubo.globalLightDirection.xyz * TBN;
    vs_out.viewDir = (worldPos.xyz - ubo.cameraPos.xyz) * TBN;
}

Fragment shader:

vec3 calculate_diffuse(vec3 color, vec3 normal)
{
    float diffuse_strength = dot(normal, -normalize(fs_in.lightDir));

    // HALF-LAMBERT
    diffuse_strength = diffuse_strength * 0.5f + 0.5f;
    diffuse_strength = clamp(diffuse_strength, 0.f, 1.f);

    return color * diffuse_strength;
}

void main()
{
    vec4 color = texture(samplerColorMap, fs_in.texcoord);

    if (ALPHA_MASK) {
        if (color.a < ALPHA_MASK_CUTOFF) {
            discard;
        }
    }

    vec3 localNormal = 2.f * texture(samplerNormalMap, fs_in.texcoord).rgb - 1.f;
    vec3 normal = normalize(localNormal);

    vec3 diffuse = calculate_diffuse(color.rgb, normal);

    outFragColor = vec4(diffuse, color.a);
}
  1. Transform the sampled normal from tangent space to world space inside the fragment shader, using the TBN matrix calculated inside the vertex shader, also with no success.

Vertex shader:

void main() 
{
    vec4 worldPos = primitive.model * vec4(inPosition, 1.0f);
    gl_Position = ubo.viewProjection * worldPos;

    vs_out.normal = inNormal;
    vs_out.tangent = inTangent;
    vs_out.bitangent = inBitangent;
    vs_out.texcoord = inTexcoord;

    vs_out.lightDir = ubo.globalLightDirection.xyz;
    vs_out.viewDir = (worldPos.xyz - ubo.cameraPos.xyz);

    vec3 T = normalize(mat3(primitive.model) * inTangent.xyz);
    vec3 B = normalize(mat3(primitive.model) * inBitangent);
    vec3 N = normalize(mat3(primitive.model) * inNormal);

    vs_out.TBN = mat3(T, B, N);
}

Fragment shader:

vec3 calculate_diffuse(vec3 color, vec3 normal)
{
    float diffuse_strength = dot(normal, -normalize(fs_in.lightDir));

    // HALF-LAMBERT
    diffuse_strength = diffuse_strength * 0.5f + 0.5f;
    diffuse_strength = clamp(diffuse_strength, 0.f, 1.f);

    return color * diffuse_strength;
}


void main()
{
    vec4 color = texture(samplerColorMap, fs_in.texcoord);

    if (ALPHA_MASK) {
        if (color.a < ALPHA_MASK_CUTOFF) {
            discard;
        }
    }

    vec3 localNormal = 2.f * texture(samplerNormalMap, fs_in.texcoord).rgb - 1.f;
    vec3 normal = normalize(localNormal * fs_in.TBN);

    vec3 diffuse = calculate_diffuse(color.rgb, normal);

    outFragColor = vec4(diffuse, color.a);
}
  1. I used Sascha Willems' shader code, but this resulted in the same seams. Sascha Willems' glTF scene rendering example Sascha Willems'vertex and fragment shader used with the above example

my github repo can be found here The mesh is imported and tangents calculated in coral_mesh::Builder::load_from_gltf() and the shaders can be found inside the shader folder

Thanks a lot in advance!

Images:

Helmet diffuse lighting with seam running across the middle

Sponza diffuse lighting with seams in the tangents


Solution

  • So, the issue was with the normal texture not having the format I thought. A normal texture should have the VkFormat "VK_FORMAT_R8G8B8_UNORM", which is what I provided to the constructor of my texture abstraction. HOWEVER, the constructor was never using this provided value and was initializing all textures with "VK_FORMAT_R8G8B8_SRGB". Fixing this issue, fixed all the seams. It was the classic image view format issue that got me and many other graphics programmers.