swiftmetalfragment-shadermetalkitmsaa

How Do I Enable MSAA for a Render to Texture iOS App


I have a working render to texture toy iOS app. The issue is it has a ton of jaggies because it is point sampled and not anti-aliasied:

enter image description here

I increased the sample count in my MTKView subclass to 4 to enable MSAA.

Here is what the relevant code looks like.

// render to texture render pass descriptor
renderPassDesc = MTLRenderPassDescriptor()
renderPassDesc.EI_configure(clearColor: MTLClearColorMake(1, 1, 1, 1), clearDepth: 1)

// my MTLRenderPassDescriptor extension convenience method
public func EI_configure(clearColor:MTLClearColor, clearDepth: Double) {

    // color
    colorAttachments[ 0 ] = MTLRenderPassColorAttachmentDescriptor()
    colorAttachments[ 0 ].storeAction = .store
    colorAttachments[ 0 ].loadAction = .clear
    colorAttachments[ 0 ].clearColor = clearColor

    // depth
    depthAttachment = MTLRenderPassDepthAttachmentDescriptor()
    depthAttachment.storeAction = .dontCare
    depthAttachment.loadAction = .clear
    depthAttachment.clearDepth = clearDepth;

}

I attach a color and depth buffer configured for MSAA to renderPassDesc:

// color
let colorDesc = MTLTextureDescriptor.texture2DDescriptor(pixelFormat:view.colorPixelFormat, width:Int(view.bounds.size.width), height:Int(view.bounds.size.height), mipmapped:false)
colorDesc.mipmapLevelCount = 1;
colorDesc.textureType = .type2DMultisample
colorDesc.sampleCount = view.sampleCount
colorDesc.usage = [.renderTarget, .shaderRead]
renderPassDesc.colorAttachments[ 0 ].texture = view.device!.makeTexture(descriptor:colorDesc)

// depth
let depthDesc = MTLTextureDescriptor.texture2DDescriptor(pixelFormat:.depth32Float, width:Int(view.bounds.size.width), height:Int(view.bounds.size.height), mipmapped:false)
depthDesc.mipmapLevelCount = 1;
depthDesc.textureType = .type2DMultisample
depthDesc.sampleCount = view.sampleCount
depthDesc.usage = .renderTarget
renderPassDesc.depthAttachment.texture = view.device!.makeTexture(descriptor:depthDesc)

In my draw loop I am getting the following error from my fragment shader that consumes the texture that was rendered into:

failed assertion Fragment Function(finalPassOverlayFragmentShader): incorrect type of texture (MTLTextureType2DMultisample) bound at texture binding at index 0 (expect MTLTextureType2D) for underlay[0]

This is the fragment shader:

fragment float4 finalPassOverlayFragmentShader(InterpolatedVertex vert [[ stage_in ]],
                                               texture2d<float> underlay [[ texture(0) ]],
                                               texture2d<float> overlay [[ texture(1) ]]) {

    constexpr sampler defaultSampler;

    float4 _F = overlay.sample(defaultSampler, vert.st).rgba;

    float4 _B = underlay.sample(defaultSampler, vert.st).rgba;

    float4 rgba = _F + (1.0f - _F.a) * _B;

    return rgba;
}

I am sure I have missed a setting somewhere but I cannot find it.

What have I missed here?

UPDATE 0

I now have MSAA happening for my 2-pass toy. The only problem is there is not much anti-aliasing happening. In fact it is hard to tell that anything has changed. Here are my latest settings

// color - multi-sampled texture target
let desc = MTLTextureDescriptor.texture2DDescriptor(pixelFormat:format, width:w, height:h, mipmapped:false)
desc.mipmapLevelCount = 1;
desc.textureType = .type2DMultisample
desc.sampleCount = view.sampleCount
desc.usage = .renderTarget
let tex:MTLTexture? = view.device!.makeTexture(descriptor:desc)

// color - point-sampled resolve-texture
let resolveDesc = MTLTextureDescriptor.texture2DDescriptor(pixelFormat:format, width:w, height:h, mipmapped:true)
let resolveTex:MTLTexture? = view.device!.makeTexture(descriptor:resolveDesc)

// depth texture target
let depthDesc = MTLTextureDescriptor.texture2DDescriptor(pixelFormat:.format, width:w, height:h, mipmapped:false)
depthDesc.mipmapLevelCount = 1;
depthDesc.textureType = .type2DMultisample
depthDesc.sampleCount = view.sampleCount
depthDesc.usage = .renderTarget
let depthTex:MTLTexture? = view.device!.makeTexture(descriptor:depthDesc)

// render pass descriptor
renderPassDesc = MTLRenderPassDescriptor()
// color
renderPassDesc.colorAttachments[ 0 ] = MTLRenderPassColorAttachmentDescriptor()
renderPassDesc.colorAttachments[ 0 ].storeAction = .storeAndMultisampleResolve
renderPassDesc.colorAttachments[ 0 ].loadAction = .clear
renderPassDesc.colorAttachments[ 0 ].clearColor = MTLClearColorMake(0.25, 0.25, 0.25, 1)
renderPassDesc.colorAttachments[ 0 ].texture = tex
renderPassDesc.colorAttachments[ 0 ].resolveTexture = resolveTex
// depth
renderPassDesc.depthAttachment = MTLRenderPassDepthAttachmentDescriptor()
renderPassDesc.depthAttachment.storeAction = .dontCare
renderPassDesc.depthAttachment.loadAction = .clear
renderPassDesc.depthAttachment.clearDepth = 1;
renderPassDesc.depthAttachment.texture = depthTex

UPDATE 1

The jaggies appear to be from the render-to-texture and not in the asset. Below is a side by side comparison. the top image is rendered using a single pass with MSAA enabled. The bottom image is rendered to texture. The jaggies are clearly visible in the bottom image

single pass enter image description here

2-pass enter image description here


Solution

  • The error is not about your render target (a.k.a. color and depth attachments). It's about a texture you're passing in via the render command encoder's fragment texture table — that is, where you're calling setFragmentTexture(_:index:). The one you're passing for index 0 is a .type2DMultisample when the shader is coded to expect .type2D, because you declared underlay as a texture2d<...>.

    Your setup for MSAA is OK for an intermediate step. You will eventually need to resolve the texture to a non-multisampled texture in order to draw it to screen. For that step (which might be for this render command encoder or a later one, depending on your needs), you need to set the storeAction for the color attachment to either .multisampleResolve or .storeAndMultisampleResolve. And you need to set the resolveTexture to a 2D texture. That could be one of your own or a drawable's texture.