Can I get the measurements for eyes using ARKit 2

Question

I am working on a project which requires size of each eye and distance between the eyes using ARKit and True depth Camera on iPhone X with iOS 12 beta 4 on Xcode 10 beta 4.

As I am new to ARKit, I am searching the solution online with very little understanding. However, I came across a demo code on apple developer portal. In that code, I tried to fetch the required values.

Also referred to the answer by @rickster

I worked on ARSCNFaceGeometry and ARFaceAnchor to fetch few values.

func update(withFaceAnchor anchor: ARFaceAnchor) {
    let faceGeometry = geometry as! ARSCNFaceGeometry
    faceGeometry.update(from: anchor.geometry)
    print("\nLook at Point :", anchor.lookAtPoint, "\nLeft :", anchor.leftEyeTransform, "\nRight :", anchor.rightEyeTransform)
}

Output:

Look at Point : float3(0.14502259, 0.2905016, 0.97809345)

Left : simd_float4x4([[0.99312633, -0.030915117, -0.11289084, 0.0)], [0.0, 0.9644885, -0.26412484, 0.0)], [0.11704737, 0.26230934, 0.9578589, 0.0)], [0.03311049, 0.028666705, 0.028539032, 1.0)]])

Right : simd_float4x4([[0.9830616, -0.047965277, -0.17688738, 0.0)], [0.0, 0.96514606, -0.26171172, 0.0)], [0.18327524, 0.25727874, 0.94879806, 0.0)], [-0.032072492, 0.028482603, 0.028181288, 1.0)]])

Questions

1. How to translate these values to make use of it and get some coordinates?

2. Will this code help me to achieve the size of each eye and distance between the eyes. If no, then what can I do to get the measurements?

Answer 1

In ARKit 2 / Xcode 10 beta 6 / iOS 12 "lookAtPoint" is nulled; but I am able to get the "leftEyeTransform" and "rightEyeTransform", and can then calculate the inter-eye distance using this code (tested on two people at varying distances and validated with a ruler!):

func update(withFaceAnchor anchor: ARFaceAnchor) {
    let faceGeometry = occlusionNode.geometry as! ARSCNFaceGeometry
    faceGeometry.update(from: anchor.geometry)
    print("\nLook at Point:", anchor.lookAtPoint, "\nLeft:", anchor.leftEyeTransform, "\nRight:", anchor.rightEyeTransform)

var leftEyePosition = SCNVector3(anchor.leftEyeTransform.columns.3.x, anchor.leftEyeTransform.columns.3.y, anchor.leftEyeTransform.columns.3.z)
var rightEyePosition = SCNVector3(anchor.rightEyeTransform.columns.3.x, anchor.rightEyeTransform.columns.3.y, anchor.rightEyeTransform.columns.3.z)
let d = distance(float3(leftEyePosition), float3(rightEyePosition))
print("\nInter-eye distance in centimeters: ", d * 100)

}

Some sample output:

Look at Point: float3(0.0, 0.0, 0.0) 
Left: simd_float4x4([[0.9997796, -0.0031222918, 0.020761598, 0.0)], [0.0, 0.98888004, 0.14871553, 0.0)], [-0.020995062, -0.14868274, 0.98866206, 0.0)], [0.031608228, 0.031428084, 0.026632207, 1.0)]]) 
Right: simd_float4x4([[0.99887705, 0.007054036, -0.04684897, 0.0)], [0.0, 0.9888536, 0.14889139, 0.0)], [0.047377057, -0.1487242, 0.98774314, 0.0)], [-0.031623494, 0.03171173, 0.02669965, 1.0)]])

Inter-eye distance in centimeters:  6.32324

NOTE:

Answer 2

One line solution for pupils distance:

simd_distance(faceAnchor.rightEyeTransform.columns.3, faceAnchor.leftEyeTransform.columns.3)

Multiple it by 100 to get the answer in centimeters. As mentioned above there is no a simple way to measure iris or pupils size, for that you need to develop an algorithm that will do the following:

1) finds circles and filters the one that describes an iris

2) use ARKit to calculate the pixel to millimeter value and use it to calculate the iris size