MTKTextureLoader saturates image

Question

I am trying to use a MTKTextureLoader to load a CGImage as a texture. Here is the original image

However after I convert that CGImage into a MTLTexture and that texture back to a CGImage it looks horrible, like this:

Here is sorta what is going on in code.

The image is loaded in as a CGImage (I have checked and that image does appear to have the full visual quality)

I have a function view() that allows me to view a NSImage by using it in a CALayer like so:

 func view() {
    .....
    imageView!.layer = CALayer()
    imageView!.layer!.contentsGravity = kCAGravityResizeAspectFill
    imageView!.layer!.contents = img
    imageView!.wantsLayer = true

So I did the following

let cg = CoolImage()
let ns = NSImage(cgImage: cg, size: Size(width: cg.width, height: cg.height))
view(image: ns)

And checked sure enough it had the full visual fidelity.

So then I loaded the cg image into a MTLTexture like so

    let textureLoader = MTKTextureLoader(device: metalState.sharedDevice!)

    let options = [
        MTKTextureLoader.Option.textureUsage: NSNumber(value: MTLTextureUsage.shaderRead.rawValue | MTLTextureUsage.shaderWrite.rawValue | MTLTextureUsage.renderTarget.rawValue),
        MTKTextureLoader.Option.SRGB: false
    ]

    return ensure(try textureLoader.newTexture(cgImage: cg, options: options))

I then converted the MTLTexture back to a UIImage like so:

    let texture = self
    let width = texture.width
    let height = texture.height
    let bytesPerRow = width * 4

    let data = UnsafeMutableRawPointer.allocate(bytes: bytesPerRow * height, alignedTo: 4)
    defer {
        data.deallocate(bytes: bytesPerRow * height, alignedTo: 4)
    }

    let region = MTLRegionMake2D(0, 0, width, height)
    texture.getBytes(data, bytesPerRow: bytesPerRow, from: region, mipmapLevel: 0)
    var buffer = vImage_Buffer(data: data, height: UInt(height), width: UInt(width), rowBytes: bytesPerRow)

    var map: [UInt8] = [0, 1, 2, 3]
    if (pixelFormat == .bgra8Unorm) {
        map = [2, 1, 0, 3]
    }
    vImagePermuteChannels_ARGB8888(&buffer, &buffer, map, 0)

    guard let colorSpace = CGColorSpace(name: CGColorSpace.genericRGBLinear) else { return nil }
    guard let context = CGContext(data: data, width: width, height: height, bitsPerComponent: 8, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue) else { return nil }
    guard let cgImage = context.makeImage() else { return nil }

    return NSImage(cgImage: cgImage, size: Size(width: width, height: height))

And viewed it.

The resulting image was quite saturated and I believe it was because of the CGImage to MTLTexture conversion which I have been fairly successful with in the past.

Please note that this texture was never rendered only converted.

You are probably wondering why I am using all of these conversions and that is a great point. My actual pipeline does not work anything like this HOWEVER it does require each of these conversion components to be working smoothly. This is not my actual use case just something to show the problem.

Answer 1

The problem here isn't the conversion from CGImage to MTLTexture. The problem is that you're assuming that the color space of the source image is linear. More likely than not, the image data is actually sRGB-encoded, so by creating a bitmap context with a generic linear color space, you're incorrectly telling CG that it should gamma-encode the image data before display, which leads to the desaturation you're seeing.

You can fix this by using the native color space of the original CGImage, or by otherwise accounting for the fact that your image data is sRGB-encoded.