Menu
I have photo with face on it.
I have carnival mask:
With this function I detect the face:
let ciImage = CIImage(cgImage: photo)
let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
let faces = faceDetector.features(in: ciImage)
if let face = faces.first as? CIFaceFeature {
}
How to detect the holes in the mask ?
How I can place the mask on the face after detecting the holes of the mask?
992
I'd probably try this approach:
Get leftEyePosition, rightEyePosition, and the faceAngle value. (All part of CIFaceFeature)
Calculate the distance in between the left and right eye.
Here is a link on how to calculate the distance: https://www.hackingwithswift.com/example-code/core-graphics/how-to-calculate-the-distance-between-two-cgpoints
Create constants with the original dimensions of the mask as well as the x and y distance to the center of one of the eyes.
With the distance of the eyes you calculate the new width of your mask proportionally.
That should get you a mask with the right size. Also calculate the new x and y distances to the center of one of eyes of the mask the same way.
Adjust all values proportionally again to fit the final intended size on the screen.
Place the mask on the photo using the coordinates of the eyes, offsetting by the mask eye to corner distance.
Use the faceAngle value to rotate the mask.
Before importing the mask into the project, convert it to a png with transparent background, remove the white background. You could do that in code, but that would be a lot of work and depending on the masks source file it might not turn out as well.
UPDATE, I've tried my solution. It's a simple iOS one screen app, just copy the code into the ViewController.swift file, add your mask as png and a photo of a face as photo.jpg into the project and it should work.
Here is a link to your photo as png if you want to try:
QPTF1.png
import UIKit
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
imageMethod()
}
func imageMethod() {
let uiMaskImage = UIImage(named: "QPTF1.png") //converted to png with transperancy before adding to the project
let maskOriginalWidth = CGFloat(exactly: 655.0)!
let maskOriginalHeight = CGFloat(exactly: 364.0)!
let maskOriginalEyeDistance = CGFloat(exactly: 230.0)! //increase or decrease value to change the final size of the mask
let maskOriginalLeftEyePossitionX = CGFloat(exactly: 203.0)! //increase or decrease to fine tune mask possition on x axis
let maskOriginalLeftEyePossitionY = CGFloat(exactly: 200.0)! //increase or decrease to fine tune mask possition on y axis
//This code assumes the image AND face orientation is always matching the same orientation!
//The code needs to be adjusted for other cases using UIImage.Orientation to get the orientation and adjusts the coordinates accordingly.
//CIDetector might also not detect faces which don't have the same orientation as the photo. Try to use CIDetectorImageOrientation to look for other orientations of no face has been detected.
//Also you might want to use other orientation points and scale values (right eye, nose etc.) in case the left eye, and left to right eye distance is not available.
//Also this code is very wordy, pretty sure it can be cut down to half the size and made simpler on many places.
let uiImageFace = UIImage(named: "photo.jpg")
let ciImageFace = CIImage(image: uiImageFace!)
let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
let faces = faceDetector.features(in: ciImageFace!)
if let face = faces.first as? CIFaceFeature {
/*
Getting the distances and angle based on the original photo
*/
let faceAngle = face.faceAngle
let rotationAngle = CGFloat(faceAngle * .pi / 180.0)
//The distance in between the eyes of the original photo.
let originalFaceEyeDistance = CGPointDistance(from: face.leftEyePosition, to: face.rightEyePosition)
/*
Adjusting the mask and its eye coordinates to fit the original photo.
*/
//Setting the scale mask : original.
let eyeDistanceScale = maskOriginalEyeDistance / originalFaceEyeDistance
//The new dimensions of the mask.
let newMaskWidth = maskOriginalWidth/eyeDistanceScale
let newMaskHeight = maskOriginalHeight/eyeDistanceScale
//The new mask coordinates of the left eye in relation to the original photo.
let newMaskLeftEyePossitionX = maskOriginalLeftEyePossitionX / eyeDistanceScale
let newMaskLeftEyePossitionY = maskOriginalLeftEyePossitionY / eyeDistanceScale
/*
Adjusting the size values to fit the desired final size on the screen.
*/
//Using the width of the screen to calculate the new scale.
let screenScale = uiImageFace!.size.width / view.frame.width
//The new final dimensions of the mask
let scaledToScreenMaskWidth = newMaskWidth / screenScale
let scaledToScreenMaskHeight = newMaskHeight / screenScale
//The new final dimensions of the photo.
let scaledToScreenPhotoHeight = uiImageFace!.size.height / screenScale
let scaledToScreenPhotoWidth = uiImageFace!.size.width / screenScale
//The new eye coordinates of the photo.
let scaledToScreenLeftEyeFacePositionX = face.leftEyePosition.x / screenScale
let scaledToScreenLeftEyeFacePositionY = (uiImageFace!.size.height - face.leftEyePosition.y) / screenScale //reversing the y direction
//The new eye to corner distance of the mask
let scaledToScreenMaskLeftEyeX = newMaskLeftEyePossitionX / screenScale
let scaledToScreenMaskLeftEyeY = newMaskLeftEyePossitionY / screenScale
//The final coordinates for the mask
let adjustedMaskLeftEyeX = scaledToScreenLeftEyeFacePositionX - scaledToScreenMaskLeftEyeX
let adjustedMaskLeftEyeY = scaledToScreenLeftEyeFacePositionY - scaledToScreenMaskLeftEyeY
/*
Showing the image on the screen.
*/
let baseImageView = UIImageView(image: uiImageFace!)
//If x and y is not 0, the mask x and y need to be adjusted too.
baseImageView.frame = CGRect(x: CGFloat(exactly: 0.0)!, y: CGFloat(exactly: 0.0)!, width: scaledToScreenPhotoWidth, height: scaledToScreenPhotoHeight)
view.addSubview(baseImageView)
let maskImageView = UIImageView(image: uiMaskImage!)
maskImageView.frame = CGRect(x: adjustedMaskLeftEyeX, y: adjustedMaskLeftEyeY, width: scaledToScreenMaskWidth, height: scaledToScreenMaskHeight)
maskImageView.transform = CGAffineTransform(rotationAngle: rotationAngle)
view.addSubview(maskImageView)
}
}
func CGPointDistanceSquared(from: CGPoint, to: CGPoint) -> CGFloat {
return (from.x - to.x) * (from.x - to.x) + (from.y - to.y) * (from.y - to.y)
}
func CGPointDistance(from: CGPoint, to: CGPoint) -> CGFloat {
return sqrt(CGPointDistanceSquared(from: from, to: to))
}
}
Result:
Here is my uncommented approach to scan for the eyes. It has still some quirks but should be a starting point.
import UIKit
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
imageMethod()
}
func imageMethod() {
struct coords {
let coord: (x: Int, y: Int)
let size: Int
}
let uiMaskImage = UIImage(named: "QPTF1.png") //converted to png with transperancy before adding to the project
let uiMaskImage2 = UIImage(named: "QPTF1.png")
let ciMaskImage2 = CIImage(image: uiMaskImage2!)
let context = CIContext(options: nil)
let cgMaskImage = context.createCGImage(ciMaskImage2!, from: ciMaskImage2!.extent)
let pixelData = cgMaskImage!.dataProvider!.data
let data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
let alphaLevel: CGFloat = 0.0 //0.0 - 1.0 set higher to allow images with partially transparent eyes, like sunglasses.
var possibleEyes: [coords] = []
let frame = 10
var detailLevel = 6
let sizeX = Int((uiMaskImage?.size.width)!)
let sizeY = Int((uiMaskImage?.size.height)!)
var points: [(x: Int, y: Int)] = []
var pointA_X = sizeX / 4
var pointA_Y = sizeY / 4
var pointB_X = sizeX / 4
var pointB_Y = sizeY * 3 / 4
var pointC_X = sizeX * 3 / 4
var pointC_Y = sizeY / 4
var pointD_X = sizeX * 3 / 4
var pointD_Y = sizeY * 3 / 4
var nextXsmaller = pointA_X / 2
var nextYsmaller = pointA_Y / 2
points.append((x: pointA_X, y: pointA_Y))
points.append((x: pointB_X, y: pointB_Y))
points.append((x: pointC_X, y: pointC_Y))
points.append((x: pointD_X, y: pointD_Y))
func transparentArea(_ x: Int, _ y: Int) -> Bool {
let pos = CGPoint(x: x, y: y)
let pixelInfo: Int = ((Int(uiMaskImage2!.size.width) * Int(pos.y)) + Int(pos.x)) * 4
let a = CGFloat(data[pixelInfo+3]) / CGFloat(255.0)
if a <= alphaLevel {
return true
} else {
return false
}
}
func createPoints(point: (x: Int, y: Int)) {
pointA_X = point.x - nextXsmaller
pointA_Y = point.y - nextYsmaller
pointB_X = point.x - nextXsmaller
pointB_Y = point.y + nextYsmaller
pointC_X = point.x + nextXsmaller
pointC_Y = point.y - nextYsmaller
pointD_X = point.x + nextXsmaller
pointD_Y = point.y + nextYsmaller
points.append((x: pointA_X, y: pointA_Y))
points.append((x: pointB_X, y: pointB_Y))
points.append((x: pointC_X, y: pointC_Y))
points.append((x: pointD_X, y: pointD_Y))
}
func checkSides(point: (x: Int, y: Int)) {
var xNeg = (val: 0, end: false)
var xPos = (val: 0, end: false)
var yNeg = (val: 0, end: false)
var yPos = (val: 0, end: false)
if transparentArea(point.x, point.y) {
xNeg.val = point.x
xPos.val = point.x
yNeg.val = point.y
yPos.val = point.y
while true {
if transparentArea(xNeg.val, point.y) {
xNeg.val -= 1
if xNeg.val <= frame {
break
}
} else {
xNeg.end = true
}
if transparentArea(xPos.val, point.y) {
xPos.val += 1
if xPos.val >= sizeX-frame {
break
}
} else {
xPos.end = true
}
if transparentArea(point.x, yNeg.val) {
yNeg.val -= 1
if yNeg.val <= frame {
break
}
} else {
yNeg.end = true
}
if transparentArea(point.x, yPos.val) {
yPos.val += 1
if yPos.val >= sizeY-frame {
break
}
} else {
yPos.end = true
}
if xNeg.end && xPos.end && yNeg.end && yPos.end {
let newEyes = coords(coord: (point.x, point.y), size: (xPos.val - xNeg.val) * (yPos.val - yNeg.val) )
possibleEyes.append(newEyes)
break
}
}
}
}
while detailLevel > 0 {
print("Run: \(detailLevel)")
for (index, point) in points.enumerated().reversed() {
//checking if the point is inside of an transparent area
checkSides(point: point)
points.remove(at: index)
if detailLevel > 1 {
createPoints(point: point)
}
}
detailLevel -= 1
nextXsmaller = nextXsmaller / 2
nextYsmaller = nextYsmaller / 2
}
possibleEyes.sort { $0.coord.x > $1.coord.x }
var rightEyes = possibleEyes[0...possibleEyes.count/2]
var leftEyes = possibleEyes[possibleEyes.count/2..<possibleEyes.count]
leftEyes.sort { $0.size > $1.size }
rightEyes.sort { $0.size > $1.size }
leftEyes = leftEyes.dropLast(Int(Double(leftEyes.count) * 0.01))
rightEyes = rightEyes.dropLast(Int(Double(leftEyes.count) * 0.01))
let sumXleft = ( leftEyes.reduce(0) { $0 + $1.coord.x} ) / leftEyes.count
let sumYleft = ( leftEyes.reduce(0) { $0 + $1.coord.y} ) / leftEyes.count
let sumXright = ( rightEyes.reduce(0) { $0 + $1.coord.x} ) / rightEyes.count
let sumYright = ( rightEyes.reduce(0) { $0 + $1.coord.y} ) / rightEyes.count
let maskOriginalWidth = CGFloat(exactly: sizeX)!
let maskOriginalHeight = CGFloat(exactly: sizeY)!
let maskOriginalLeftEyePossitionX = CGFloat(exactly: sumXleft)!
let maskOriginalLeftEyePossitionY = CGFloat(exactly: sumYleft)!
let maskOriginalEyeDistance = CGPointDistance(from: CGPoint(x: sumXright, y: sumYright), to: CGPoint(x: sumXleft, y: sumYleft))
//This code assumes the image AND face orientation is always matching the same orientation!
//The code needs to be adjusted for other cases using UIImage.Orientation to get the orientation and adjusts the coordinates accordingly.
//CIDetector might also not detect faces which don't have the same orientation as the photo. Try to use CIDetectorImageOrientation to look for other orientations of no face has been detected.
//Also you might want to use other orientation points and scale values (right eye, nose etc.) in case the left eye, and left to right eye distance is not available.
//Also this code is very wordy, pretty sure it can be cut down to half the size and made simpler on many places.
let uiImageFace = UIImage(named: "photo3.jpg")
let ciImageFace = CIImage(image: uiImageFace!)
let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
let faces = faceDetector.features(in: ciImageFace!)
if let face = faces.first as? CIFaceFeature {
/*
Getting the distances and angle based on the original photo
*/
let faceAngle = face.faceAngle
let rotationAngle = CGFloat(faceAngle * .pi / 180.0)
//The distance in between the eyes of the original photo.
let originalFaceEyeDistance = CGPointDistance(from: face.leftEyePosition, to: face.rightEyePosition)
/*
Adjusting the mask and its eye coordinates to fit the original photo.
*/
//Setting the scale mask : original.
let eyeDistanceScale = maskOriginalEyeDistance / originalFaceEyeDistance
//The new dimensions of the mask.
let newMaskWidth = maskOriginalWidth/eyeDistanceScale
let newMaskHeight = maskOriginalHeight/eyeDistanceScale
//The new mask coordinates of the left eye in relation to the original photo.
let newMaskLeftEyePossitionX = maskOriginalLeftEyePossitionX / eyeDistanceScale
let newMaskLeftEyePossitionY = maskOriginalLeftEyePossitionY / eyeDistanceScale
/*
Adjusting the size values to fit the desired final size on the screen.
*/
//Using the width of the screen to calculate the new scale.
let screenScale = uiImageFace!.size.width / view.frame.width
//The new final dimensions of the mask
let scaledToScreenMaskWidth = newMaskWidth / screenScale
let scaledToScreenMaskHeight = newMaskHeight / screenScale
//The new final dimensions of the photo.
let scaledToScreenPhotoHeight = uiImageFace!.size.height / screenScale
let scaledToScreenPhotoWidth = uiImageFace!.size.width / screenScale
//The new eye coordinates of the photo.
let scaledToScreenLeftEyeFacePositionX = face.leftEyePosition.x / screenScale
let scaledToScreenLeftEyeFacePositionY = (uiImageFace!.size.height - face.leftEyePosition.y) / screenScale //reversing the y direction
//The new eye to corner distance of the mask
let scaledToScreenMaskLeftEyeX = newMaskLeftEyePossitionX / screenScale
let scaledToScreenMaskLeftEyeY = newMaskLeftEyePossitionY / screenScale
//The final coordinates for the mask
let adjustedMaskLeftEyeX = scaledToScreenLeftEyeFacePositionX - scaledToScreenMaskLeftEyeX
let adjustedMaskLeftEyeY = scaledToScreenLeftEyeFacePositionY - scaledToScreenMaskLeftEyeY
/*
Showing the image on the screen.
*/
let baseImageView = UIImageView(image: uiImageFace!)
//If x and y is not 0, the mask x and y need to be adjusted too.
baseImageView.frame = CGRect(x: CGFloat(exactly: 0.0)!, y: CGFloat(exactly: 0.0)!, width: scaledToScreenPhotoWidth, height: scaledToScreenPhotoHeight)
view.addSubview(baseImageView)
let maskImageView = UIImageView(image: uiMaskImage!)
maskImageView.frame = CGRect(x: adjustedMaskLeftEyeX, y: adjustedMaskLeftEyeY, width: scaledToScreenMaskWidth, height: scaledToScreenMaskHeight)
maskImageView.transform = CGAffineTransform(rotationAngle: rotationAngle)
view.addSubview(maskImageView)
}
}
func CGPointDistanceSquared(from: CGPoint, to: CGPoint) -> CGFloat {
return (from.x - to.x) * (from.x - to.x) + (from.y - to.y) * (from.y - to.y)
}
func CGPointDistance(from: CGPoint, to: CGPoint) -> CGFloat {
return sqrt(CGPointDistanceSquared(from: from, to: to))
}
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
