Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Slow face detection Firebase MLKit

Started playing around with MLKit face detectors with the front-facing camera, but it's really slow on processing faces

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

    print("Picture at ", Date())


    let visionImage = VisionImage(buffer: sampleBuffer)
    visionImage.metadata = metadata


    faceDetector?.detect(in: visionImage) { (faces, error) in
        guard error == nil, let faces = faces, !faces.isEmpty else {
            // Error. You should also check the console for error messages.
            let errorString = error?.localizedDescription
            print("Face detection failed with error: \(errorString)")
            return
        }

  }

Where am i going wrong?

like image 426
Itumeleng Mabote Avatar asked May 21 '18 04:05

Itumeleng Mabote


People also ask

Is Mlkit offline?

ML Kit's APIs all run on-device, allowing for real-time use cases where you want to process a live camera stream for example. This also means that the functionality is available offline. All ML Kit APIs require Android API level 19 or higher.

What is ML kit in firebase?

ML Kit is a mobile SDK that brings Google's machine learning expertise to Android and iOS apps in a powerful yet easy-to-use package. Whether you're new or experienced in machine learning, you can implement the functionality you need in just a few lines of code.


1 Answers

There are a few things you can try to speed up the detection:

  1. Build your app in the release mode (optimized), as opposed to the debug mode.

  2. When creating your faceDetector, please make sure to use a VisionFaceDetectorOptions and set its isTrackingEnabled to true.

  3. When setting up your AVCaptureVideoDataOutput, please add the following key-value pair to its videoSettings:

key: kCVPixelBufferPixelFormatTypeKey

value: kCVPixelFormatType_32BGRA

like image 63
Dong Chen Avatar answered Sep 30 '22 04:09

Dong Chen