Detect Eye Blink Using Google Vision API

Question

i'm using the vision API for face detection, now i want to implement eye blink but still vision api detect eye while one eye is off.

please help me how to implement eye blink feature.

Answer 1

The "eye open probability" values from the face are the key to detecting the blink. In addition, you can use a Tracker to keep track of the eye state over time, to detect the sequence of events that indicate a blink:

both eyes open -> both eyes closed -> both eyes open

Here's an example tracker:

public class BlinkTracker extends Tracker<Face> {
  private final float OPEN_THRESHOLD = 0.85;
  private final float CLOSE_THRESHOLD = 0.15;

  private int state = 0;

  public void onUpdate(Detector.Detections<Face> detections, Face face) {
    float left = face.getIsLeftEyeOpenProbability();
    float right = face.getIsRightEyeOpenProbability();
    if ((left == Face.UNCOMPUTED_PROBABILITY) ||
        (right == Face.UNCOMPUTED_PROBABILITY)) {
      // At least one of the eyes was not detected.
      return;
    }

    switch (state) {
      case 0:
        if ((left > OPEN_THRESHOLD) && (right > OPEN_THRESHOLD)) {
          // Both eyes are initially open
          state = 1;
        }
        break;

        case 1:
          if ((left < CLOSE_THRESHOLD) && (right < CLOSE_THRESHOLD)) {
            // Both eyes become closed
            state = 2;
          }
          break;

        case 2:
          if ((left > OPEN_THRESHOLD) && (right > OPEN_THRESHOLD)) {
            // Both eyes are open again
            Log.i("BlinkTracker", "blink occurred!");
            state = 0;
          }
        break;
    }
  }

}

Note that you need to also enable "classifications" in order to have the detector indicate if eyes are open/closed:

FaceDetector detector = new FaceDetector.Builder(context)
    .setClassificationType(FaceDetector.ALL_CLASSIFICATIONS)
    .build();

The tracker is then added as a processor for receiving face updates over time from the detector. For example, this configuration would be used to track whether the largest face in view has blinked:

detector.setProcessor(
    new LargestFaceFocusingProcessor(detector, new BlinkTracker()));

Alternatively, you could use a MultiProcessor instead of LargestFaceFocusingProcessor if you are interested in detecting blinks for all faces (not just the largest face).