Android MLKit scan QRcode only on center of screen

Question

I implemented firebase MLKit to scan QRCode, it is scanning, but it is scanning all QRcode on screen. I need to scan only QRCode that are captured on the center (has arrow ImageView on the center), how can I do it?

I tried to crop on the Analisys function (inside analysisUseCase?.setAnalyzer)

imageProxy.cropRect()

I tried to crop on the processImageProxy function. But with no success, I think that I can't crop with this

class QrcodeScanner(
        private val onQrCapture: Barcode.() -> Unit,
        private val onFailure: Throwable.() -> Unit,
        private val lifecycleOwner: LifecycleOwner,
        private val context: Context,
        private val previewView: PreviewView
) {

    private var cameraSelector: CameraSelector = CameraSelector.Builder()
            .requireLensFacing(CameraSelector.LENS_FACING_BACK)
            .build()
    private var cameraProvider: ProcessCameraProvider? = null
    private var previewUseCase: Preview? = null
    private var analysisUseCase: ImageAnalysis? = null

    fun startCamera() {
        val cameraProviderFuture =
                ProcessCameraProvider.getInstance(context)
        cameraProviderFuture.addListener(
                {
                    runCatching {
                        val provider = cameraProviderFuture.get()
                        cameraProvider = provider
                        startPreview()
                        startAnalysis()
                    }.onFailure {
                        onFailure(it)
                    }
                },
                ContextCompat.getMainExecutor(context)
        )
    }

    private fun startPreview() {
        if (previewUseCase != null) {
            cameraProvider?.unbind(previewUseCase)
        }

        previewUseCase = Preview.Builder()
                .setTargetRotation(previewView.display.rotation)
                .build()
        previewUseCase?.setSurfaceProvider(previewView.surfaceProvider)

        runCatching {
            cameraProvider?.bindToLifecycle(lifecycleOwner,
                    cameraSelector,
                    previewUseCase
            )
        }.onFailure {
            onFailure(it)
        }
    }

    private fun startAnalysis() {
        val options = BarcodeScannerOptions.Builder()
             .setBarcodeFormats(Barcode.FORMAT_QR_CODE)
             .build()
        val barcodeScanner: BarcodeScanner = BarcodeScanning.getClient(options)

        if (cameraProvider == null) {
            return
        }
        if (analysisUseCase != null) {
            cameraProvider?.unbind(analysisUseCase)
        }

        analysisUseCase = ImageAnalysis.Builder()
                .setTargetRotation(previewView.display.rotation)
                .build()

        val cameraExecutor = Executors.newSingleThreadExecutor()

        analysisUseCase?.setAnalyzer(cameraExecutor, ImageAnalysis.Analyzer { imageProxy ->
            processImageProxy(barcodeScanner, imageProxy)
        })

        runCatching {
            cameraProvider?.bindToLifecycle(lifecycleOwner,
                    cameraSelector,
                    analysisUseCase
            )
        }.onFailure {
            onFailure(it)
        }

    }

    @SuppressLint("UnsafeExperimentalUsageError")
    private fun processImageProxy(
            barcodeScanner: BarcodeScanner,
            imageProxy: ImageProxy
    ) {
        runCatching {
            val img = imageProxy.image
            if (img != null) {
                val inputImage =
                        InputImage.fromMediaImage(img, imageProxy.imageInfo.rotationDegrees)

                barcodeScanner.process(inputImage)
                        .addOnSuccessListener { barcodes ->
                            barcodes.forEach {
                                onQrCapture(it)
                            }
                        }
                        .addOnFailureListener {
                            onFailure(it)
                        }.addOnCompleteListener {
                            imageProxy.close()
                        }
            } else {
                throw Exception("Falha ao processar a imagem")
            }
        }.onFailure {
            onFailure(it)
        }

    }

}

Answer 1

imageProxy.cropRect() only put a metadata in the image, but not do the cropping operation. For static image, you could convert to bitmap, and crop. A better way is to filter out the results when you have all the returned barcode with the detected barcode boundingbox.

Within MLKit, we are adding supports for image cropping.

Answer 2

You can amend onSuccess() in BarcodeScannerProcessor class to check the barcode's four corners position. If it is within scan region, return true from onSuccess(), as follows:

protected void onSuccess(List<Barcode> barcodes, GraphicOverlay graphicOverlay) {
    if (barcodes.isEmpty()) {
        Log.v(MANUAL_TESTING_LOG, "No barcode has been detected");
    }

    for (int i = 0; i < barcodes.size(); ++i) {
        Barcode barcode = barcodes.get(i);
        BarcodeGraphic barcodeGraphic = new BarcodeGraphic(graphicOverlay, barcode);

        // check whether within valid scan area, translateX and translateY is to convert to screen position in pixel
        if (barcode.getBoundingBox() != null &&
                withinScanArea(
                    barcodeGraphic.translateY(barcode.getBoundingBox().top),
                    barcodeGraphic.translateX(barcode.getBoundingBox().left),
                    barcodeGraphic.translateY(barcode.getBoundingBox().bottom),
                    barcodeGraphic.translateX(barcode.getBoundingBox().right)
                )
        ) {
            graphicOverlay.add(barcodeGraphic);

            if (barcode.getRawValue() != null && !barcode.getRawValue().isEmpty()) {
                exchangeScannedData.sendScannedCode(barcode.getRawValue());
            }
        }
    }
}

scanAreaTop, etc. are local variables with values according to your scan view

private boolean withinScanArea(float top, float left, float bottom, float right) {
    return top > scanAreaTop &&
           left > scanAreaLeft &&
           bottom < scanAreaBottom &&
           right < scanAreaRight;
}