iOS framework: MTLLibrary newFunctionWithName fails when cikernel flag is set on client code

Question

I have an iOS framework using Metal; it sets up a pipeline descriptor as follows:

...
_metalVars.mtlLibrary = [_metalVars.mtlDevice newLibraryWithFile:libraryPath error:&error];

id <MTLFunction> vertexFunction = [_metalVars.mtlLibrary newFunctionWithName:@"vertexShader"];
id <MTLFunction> fragmentFunction = [_metalVars.mtlLibrary newFunctionWithName:@"fragmentShader"];

MTLRenderPipelineDescriptor *pipelineStateDescriptor = [[MTLRenderPipelineDescriptor alloc] init];

MTLRenderPipelineDescriptor *pipelineStateDescriptor = [[MTLRenderPipelineDescriptor alloc] init];
pipelineStateDescriptor.label = @"MyPipeline";
pipelineStateDescriptor.vertexFunction = vertexFunction;
pipelineStateDescriptor.fragmentFunction = fragmentFunction;
pipelineStateDescriptor.vertexDescriptor = mtlVertexDescriptor;
pipelineStateDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormatRGBA8Unorm; 

_metalVars.mtlPipelineState = [_metalVars.mtlDevice newRenderPipelineStateWithDescriptor:pipelineStateDescriptor error:&error];
...

This framework is used by an iOS app, and has been working fine until the flag

-fcikernel

was set under Other Metal Compiler Flags, and

-cikernel

underOther Metal Linker Flags, in the client app's Build Settings. (flags set on client, per https://developer.apple.com/documentation/coreimage/cikernel/2880194-kernelwithfunctionname.)

With those changes in the client app, the above framework code snippet now fails at the last line of the above code snippet, on the call to newRenderPipelineStateWithDescriptor with the error

validateWithDevice:2556: failed assertion `Render Pipeline Descriptor Validation
vertexFunction must not be nil

and I've verified that the framework is now returning nil for the vertex function at

id <MTLFunction> vertexFunction = [_metalVars.mtlLibrary newFunctionWithName:@"vertexShader"];

but works fine if I remove the flags on the client app.

Questions: 1.) I've understood that metal shaders were precompiled; why would changing a build setting on the client code cause this runtime failure in the unchanged framework? 2.) How can this be resolved?

Answer 1

Those flags are meant to be used when compiling Core Image kernels written in Metal. They change the way the code is translated in a way that is not compatible with "regular" Metal shader code.

The problem with setting the Core Image flags on the project level is that all .metal files are affected—regardless of whether they contain Core Image kernels or regular shader code.

The best way around this is by using a different file extension for files containing Core Image Metal kernels, like .ci.metal. Then custom build rules can be used to compile those kernels separately from the rest of the Metal codebase. I recommend this session from WWDC 2020 which describes the process in detail.