scenekitmetalmetalkitscnscenemtkview

Interoperate MTKView (MetalKit) and SceneKit


I am using MetalKit and have a complex rendering pipeline. The results are rendered into a MTKView.

Now I would like to feed the contents of the MTKView to a SCNScene and using a SCNCamera to perform post-process effects like HDR.

How is this possible?

I do not want general directions. I want specific calls if possible.


Solution

  • You should ideally perform your post-process as part of your Metal rendering pipeline. The procedure you are suggesting requires unnecessary resources since you will be rendering a 2D plane in 3D in SceneKit just to apply some HDR effects.

    Nevertheless, you can achieve what you want by rendering your Metal pipeline output to a texture and then simply applying it to a plane in SceneKit.

    First assign your texture:

    plane.materials.first?.diffuse.contents = offscreenTexture

    Then override your SceneKit rendering to your Metal rendering loop:

    func renderer(_ renderer: SCNSceneRenderer, willRenderScene scene: SCNScene, atTime time: TimeInterval) {
        doRender()
    }
    

    Then perform your Metal rendering with the texture as target, and once that is done you render your SceneKit scene:

    func doRender() {
        //rendering to a MTLTexture, so the viewport is the size of this texture
        let viewport = CGRect(x: 0, y: 0, width: CGFloat(textureSizeX), height: CGFloat(textureSizeY))
    
        //write to offscreenTexture, clear the texture before rendering using green, store the result
        let renderPassDescriptor = MTLRenderPassDescriptor()
        renderPassDescriptor.colorAttachments[0].texture = offscreenTexture
        renderPassDescriptor.colorAttachments[0].loadAction = .clear
        renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(0, 1, 0, 1.0); //green
        renderPassDescriptor.colorAttachments[0].storeAction = .store
    
        let commandBuffer = commandQueue.makeCommandBuffer()
    
        // reuse scene1 and the current point of view
        renderer.scene = scene1
        renderer.pointOfView = scnView1.pointOfView
        renderer.render(atTime: 0, viewport: viewport, commandBuffer: commandBuffer, passDescriptor: renderPassDescriptor)
    
        commandBuffer.commit()
    }`
    

    Full example project:

    https://github.com/lachlanhurst/SceneKitOffscreenRendering