Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

UIImage created from MTKView results in color/opacity differences

When I capture the contents of an MTKView into a UIImage, the resulting image looks qualitatively different, as shown below:

MTKView vs UIImageView of the same

The code I use to generate the UIImage is as follows:

let kciOptions = [kCIContextWorkingColorSpace: CGColorSpace(name: CGColorSpace.sRGB)!,
                         kCIContextOutputPremultiplied: true,
                         kCIContextUseSoftwareRenderer: false] as [String : Any]
let lastDrawableDisplayed = self.currentDrawable! // needed to hold the last drawable presented to screen
drawingUIView.image = UIImage(ciImage: CIImage(mtlTexture: lastDrawableDisplayed.texture, options: kciOptions)!)

Since I don't modify the ciImage orientation (.oriented(CGImagePropertyOrientation.downMirrored)) the resulting image is upside down, as shown in the image above. I leave the mirrored orientation as is so I can point out the color differences between the two image captures.

No matter how I change the kciOptions parameters, (say, even changing the colorspace to grayscale) I'm not seeing any changes in the resulting UIImage, which appears much more dim/desaturated than the original. Does anybody have any suggestions for how I can accurately capture what I'm drawing on MTKView to an UIImage? Any suggestions would be much appreciated.

Below are my MTKView settings which may prove relevant:

let renderPipelineDescriptor = MTLRenderPipelineDescriptor()
renderPipelineDescriptor.vertexFunction = vertexProgram
renderPipelineDescriptor.sampleCount = self.sampleCount
renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm
renderPipelineDescriptor.colorAttachments[0].isBlendingEnabled = true
renderPipelineDescriptor.colorAttachments[0].rgbBlendOperation = .add
renderPipelineDescriptor.colorAttachments[0].alphaBlendOperation = .add
renderPipelineDescriptor.colorAttachments[0].sourceRGBBlendFactor = .sourceAlpha         renderPipelineDescriptor.colorAttachments[0].destinationRGBBlendFactor = .oneMinusSourceAlpha            renderPipelineDescriptor.colorAttachments[0].sourceAlphaBlendFactor = .sourceAlpha              renderPipelineDescriptor.colorAttachments[0].destinationAlphaBlendFactor = .oneMinusSourceAlpha
self.isOpaque = false // makes MTKView bg transparent
like image 593
Plutovman Avatar asked Mar 11 '18 19:03

Plutovman


2 Answers

I've seen this issue in a couple of posts, but no clear answer. Here is what I've found:

For starters,

renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm

should really just be set to the MTKView's native pixel format

renderPipelineDescriptor.colorAttachments[0].pixelFormat = self.colorPixelFormat

Secondly, when I set the CIImage's options:

let kciOptions = [kCIContextWorkingColorSpace: CGColorSpace(name: CGColorSpace.sRGB)!,
                         kCIContextOutputPremultiplied: true,
                         kCIContextUseSoftwareRenderer: false] as [String : Any]

It didn't matter what I set kCIContextWorkingColorSpace to, I never saw any visual difference regardless of what I used. The property I really needed to set is called KCIImageColorSpace. So the updated kciOptions looks like:

let kciOptions = [kCIImageColorSpace: CGColorSpaceCreateDeviceRGB(),
                      kCIContextOutputPremultiplied: true,
                      kCIContextUseSoftwareRenderer: false] as [String : Any]

In a similar way of using the view's native pixel format, calling CGColorSpaceCreateDeviceRGB() creates an RGB colorspace that is specific to the device being used.

like image 156
Plutovman Avatar answered Dec 03 '22 18:12

Plutovman


Your CGColorSpace is .sRGB but your renderPipelineDescriptor's pixelFormat is .bgra8Unorm. Try changing that line to:

renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm_srgb
like image 31
hartw Avatar answered Dec 03 '22 19:12

hartw