Started playing around with MLKit face detectors with the front-facing camera, but it's really slow on processing faces
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
print("Picture at ", Date())
let visionImage = VisionImage(buffer: sampleBuffer)
visionImage.metadata = metadata
faceDetector?.detect(in: visionImage) { (faces, error) in
guard error == nil, let faces = faces, !faces.isEmpty else {
// Error. You should also check the console for error messages.
let errorString = error?.localizedDescription
print("Face detection failed with error: \(errorString)")
return
}
}
Where am i going wrong?
ML Kit's APIs all run on-device, allowing for real-time use cases where you want to process a live camera stream for example. This also means that the functionality is available offline. All ML Kit APIs require Android API level 19 or higher.
ML Kit is a mobile SDK that brings Google's machine learning expertise to Android and iOS apps in a powerful yet easy-to-use package. Whether you're new or experienced in machine learning, you can implement the functionality you need in just a few lines of code.
There are a few things you can try to speed up the detection:
Build your app in the release mode (optimized), as opposed to the debug mode.
When creating your faceDetector, please make sure to use a VisionFaceDetectorOptions and set its isTrackingEnabled to true.
When setting up your AVCaptureVideoDataOutput, please add the following key-value pair to its videoSettings:
key: kCVPixelBufferPixelFormatTypeKey
value: kCVPixelFormatType_32BGRA
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With