How to chain filters in Metal for iOS?

Question

I completed this tutorial by Simon Gladman (@flexmonkey) to capture images from AVFoundation and apply a filter to the output. However, I'm struggling to find a way to replace the blur filter with my own compute shader. In other words, I need to concatenate my custom shader after the YCbCrColorConversion filter mentioned there.

let commandBuffer = commandQueue.makeCommandBuffer()
let commandEncoder = commandBuffer.makeComputeCommandEncoder()
// pipelineState has compiled YCbCrColorConversion filter
commandEncoder.setComputePipelineState(pipelineState)
commandEncoder.setTexture(ytexture, at: 0)
commandEncoder.setTexture(cbcrTexture, at: 1)
commandEncoder.setTexture(drawable.texture, at: 2) // out texture

commandEncoder.dispatchThreadgroups(threadGroups,threadsPerThreadgroup: threadGroupCount)
commandEncoder.endEncoding()

let inPlaceTexture = UnsafeMutablePointer<MTLTexture> .allocate(capacity: 1)
        inPlaceTexture.initialize(to: drawable.texture)

// How to replace this blur with my own filter?????
blur.encodeToCommandBuffer(commandBuffer, inPlaceTexture: inPlaceTexture, fallbackCopyAllocator: nil)

commandBuffer.presentDrawable(drawable)
commandBuffer.commit();

Should I create a new commandBuffer, commandEncoder and a separate pipelineState that compiles the second kernel function? This would take the output of the first filter as an input to the second. Is there a more efficient way to do this, or this is optimal?

I'm a beginner with Metal, so any explanations on how the pipeline works are highly appreciated.

Answer 1

You don't need to create a new command buffer or another compute encoder, but you do need to create a compute pipeline state that uses your own kernel function. You should do this once, during initialization, wherever you're currently creating the YCbCr conversion pipeline state.

To chain the effects together, you'll need to create an intermediate texture that acts as the output texture of the YCbCr conversion and the input of your kernel. The drawable texture will then be the output texture of your kernel function. You can dispatch the work for your own kernel just as you're currently dispatching the work for the YCbCr conversion (i.e. with the same number of threads per threadgroup and threadgroup count).

The intermediate texture should probably be of the same dimensions and format as the drawable. You can create it lazily and hold a reference to it, re-creating it when the drawable size changes.