I am writing an AR application with which im testing different shaders for a university project. Some of them are already available via the Metal Performance Shaders (right now I am only using Sobel and Gaussian Blur on an image. I also have a couple of custom kernel shaders written. First I do a pass over the original image, i.e. ARFrame in my case, with a shader A
to distort it and in the second pass I apply another shader B
to try and fix the previous damage again.
Original frame --> A --> B (result)
Plain and simple and it works great with my custom shaders (they aren't too sophisticated yet). Now, when I replace either A
or B
with an MPS it also works like a charm. But once I replace both A
and B
with an MPS, the program crashes and I get the following error message:
failed assertion `[MPSImageGaussianBlur encodeToCommandBuffer:sourceTexture:destinationTexture: can not operate in place.'
Since this is an AR application, I am assigning a callback function to PostProcessContext
and handle everything else in there. The code looks like below (I obviously stripped some boilerplate out but I hope you get the idea of what I am trying to accomplish).
func initShaders() {
// makeFunction and obtain computePipelineState
// load textures
// initialize buffers
renderCallbacks.postProcess = { context in
for (i, shaderDescriptor) in shaders.enumerated() {
// ======================================================================================== //
// If the current shader is a MPS we should be able to directly run it
if shaderDescriptor.shader.type == .metalPerformanceShader {
let gaussianBlur = MPSImageGaussianBlur(device: context.device, sigma: sigma)
gaussianBlur.encode(
commandBuffer: context.commandBuffer,
sourceTexture: context.sourceColorTexture,
destinationTexture: context.targetColorTexture
)
continue
}
// ======================================================================================== //
// ======================================================================================== //
// Otherwise use our custom shader with the regular workload
guard let encoder = context.commandBuffer.makeComputeCommandEncoder(descriptor: computePassDescriptor) else {
continue
}
encoder.setComputePipelineState(computePipelineState!)
encoder.setTexture(context.sourceColorTexture, index: 0)
encoder.setTexture(context.targetColorTexture, index: 1)
// Assign some arguments
...
encoder.setBuffer(argumentBuffer, offset: MemoryLayout<Float>.size * j, index: j)
...
// Assign some textures
...
encoder.setTexture(self.loadedTextures[name], index: i)
...
// Dispatch the threadgroups
let threadsPerThreadgroup = MTLSize(...)
let threadgroupsPerGrid = MTLSize(...)
encoder.dispatchThreadgroups(threadgroupsPerGrid, threadsPerThreadgroup: threadsPerThreadgroup)
// end the encoding and proceed with the loop to the second pass
encoder.endEncoding()
// ======================================================================================== //
}
// Does not need to be run, p
// context.commandBuffer.commit()
}
}
The problem was not within the above code but the way I handled the processing which looks like this:
renderCallbacks.postProcess = { context in
var context = context
let correctionCallback = self.postProcessCallbacks[.correction]
let simulationCallback = self.postProcessCallbacks[.simulation]
if correctionCallback != nil {
correctionCallback!!(context)
}
if simulationCallback != nil {
simulationCallback!!(context)
}
}
Between the two if's I needed to use a temporary, intermediate texture to store everything from the callback and then pass it into the second pass.