ioscamerashadermetalaspect-ratio

Metal Shader Stretching Camera Preview Vertically While Applying Grayscale Filter In iOS Swift


I'm building an iOS app where I apply a grayscale filter to a live camera feed using Metal shaders. The grayscale effect works fine, but when I apply it, the camera preview is stretched vertically. Below is the code for my Metal shader and the camera preview setup.

Shader.metal

#include <metal_stdlib>
using namespace metal;

typedef struct {
    float4 renderedCoordinate [[position]];
    float2 textureCoordinate;
} TextureMappingVertex;

vertex TextureMappingVertex mapTexture(unsigned int vertex_id [[vertex_id]]) {
    float4x4 renderedCoordinates = float4x4(
        float4( -1.0, -1.0, 0.0, 1.0 ),
        float4(  1.0, -1.0, 0.0, 1.0 ),
        float4( -1.0,  1.0, 0.0, 1.0 ),
        float4(  1.0,  1.0, 0.0, 1.0 )
    );

    float4x2 textureCoordinates = float4x2(
        float2( 0.0, 1.0 ),
        float2( 1.0, 1.0 ),
        float2( 0.0, 0.0 ),
        float2( 1.0, 0.0 )
    );

    TextureMappingVertex outVertex;
    outVertex.renderedCoordinate = renderedCoordinates[vertex_id];
    outVertex.textureCoordinate = textureCoordinates[vertex_id];
    
    return outVertex;
}

fragment half4 displayTexture(TextureMappingVertex mappingVertex [[stage_in]],
                              texture2d<float, access::sample> texture [[texture(0)]]) {
    constexpr sampler s(address::clamp_to_edge, filter::linear);
    float4 color = texture.sample(s, mappingVertex.textureCoordinate);
    float grayscale = (color.r + color.g + color.b) / 3.0;
    return half4(grayscale, grayscale, grayscale, color.a);
}

Issue:

Additional Info:

Objective:


Solution

  • You can make size of float4x2 textureCoordinates according to screenHeight and ScreenWidth just pass that both to shader vertex function and than according to aspect ratio of screen change float4x2 textureCoordinates

    float screenWidth = screenParams.screenWidth;
    float screenHeight =  screenParams.screenHeight;
    float screenAspectRatio = (screenWidth / screenHeight) + 0.3;
    
    float2 topLeft = float2(0.0, 1.0);
    float2 topRight = float2(1.0, 1.0);
    float2 bottomLeft = float2(0.0, 0.0);
    float2 bottomRight = float2(1.0, 0.0);
    
    topLeft.x *= screenAspectRatio;
    topRight.x *= screenAspectRatio;
    bottomLeft.x *= screenAspectRatio;
    bottomRight.x *= screenAspectRatio;
    
    float4x2 textureCoordinates = float4x2(topLeft, topRight, bottomLeft, bottomRight);
    

    After this it's not going to Strech anymore