ML Kit Barcode scanning: Invalid image data size

Question

I would like to detect a barcode within a captured image. I capture an image using android's camera2. Following this, the image's metadata is retrieved and the image is saved to the device. The metadata is all passed along to the next activity, which is where the application attempts to detect a barcode.

This next activity creates a byte[] from the File saved previously. Next, the relevant FirebaseVision objects are created using the data passed with the intent. Finally, the application attempts to call the detectInImage() method, where an error is thrown:

"java.lang.IllegalArgumentException: Invalid image data size."

I suspect this is from the captured image being too large, however I cannot seem to figure out how to capture a smaller image, and I also cannot find anything in the reference documentation regarding the maximum size allowed. Information regarding this error and how to solve it would be very much appreciated. Below is what I believe to be the relevant code.

private final ImageReader.OnImageAvailableListener onImageAvailableListener
        = new ImageReader.OnImageAvailableListener() {
    @Override
    public void onImageAvailable(ImageReader imageReader) {
        try{
            // Semaphore ensures date is recorded before starting next activity
            storeData.acquire();
            Image resultImg = imageReader.acquireNextImage(); // Image from camera
            imgWidth = resultImg.getWidth();
            imgHeight = resultImg.getHeight();
            ByteBuffer buffer = resultImg.getPlanes()[0].getBuffer();
            data = new byte[buffer.remaining()]; // Byte array with the images data
            buffer.get(data);
            String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());

            // Note: mediaFile directs to Pictures/"ThisProject" folder
            File media = new File(mediaFile.getPath() +
                    File.separator + "IMG_" + timeStamp + ".jpg");

            // Saving the image
            FileOutputStream fos = null;
            try {
                fos = new FileOutputStream(media);
                fos.write(data);
                uri = Uri.fromFile(media);
            } catch (IOException e) {
                Log.e(TAG, e.getMessage());
            } finally {
                if (fos != null) {
                    try {
                        fos.close();
                    } catch (IOException e) {
                        Log.e(TAG, e.getMessage());
                    }
                }
            }
            resultImg.close();
        } catch (InterruptedException e) {
            Log.e(TAG, e.getMessage());
        }
        storeData.release();

    }
};

This essentially retrieves the image height & width, then writes it to a file. The data sent to the next activity consists of the: Image width, Image height, Image rotation, and the Uri directing to the file.

Using this, I try to detect a barcode using Firebase ML Kit:

// uri is the uri referencing the saved image
File f = new File(uri.getPath());
data = new byte[(int) f.length()];

try{
        BufferedInputStream bis = new BufferedInputStream(new FileInputStream(f));
        DataInputStream dis = new DataInputStream(bis);
        dis.readFully(data);
    } catch (IOException e) {
        Log.e(TAG, e.getMessage());
    }

FirebaseVisionBarcodeDetectorOptions options = new FirebaseVisionBarcodeDetectorOptions.Builder().setBarcodeFormats(
            FirebaseVisionBarcode.FORMAT_QR_CODE,
            FirebaseVisionBarcode.FORMAT_DATA_MATRIX
    ).build();

FirebaseVisionBarcodeDetector detector = FirebaseVision.getInstance().getVisionBarcodeDetector(options);
FirebaseVisionImage image;

int rotationResult;
        switch (imgRotation) {
            case 0: {
                rotationResult = FirebaseVisionImageMetadata.ROTATION_0;
                break;
            }
            case 90: {
                rotationResult = FirebaseVisionImageMetadata.ROTATION_90;
                break;
            }
            case 180: {
                rotationResult = FirebaseVisionImageMetadata.ROTATION_180;
                break;
            }
            case 270: {
                rotationResult = FirebaseVisionImageMetadata.ROTATION_270;
                break;
            }
            default: {
                rotationResult = FirebaseVisionImageMetadata.ROTATION_0;
                break;
            }
        }

FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
            .setWidth(imgWidth)
            .setHeight(imgHeight)
            .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
            .setRotation(rotationResult)
            .build();

image = FirebaseVisionImage.fromByteArray(data, metadata);

Task<List<FirebaseVisionBarcode>> result = detector.detectInImage(image)

Answer 1

A few things.

1. Your image format should not be NV21 if you use camera2. See here for all camera2 supported image formats: https://developer.android.com/reference/android/media/Image#getFormat() Your byte[] is not NV21 and you specified IMAGE_FORMAT_NV21 and led to the error

2. Most intuitive integration with camera2 is like below:

   • Specify JPEG format when you instantiate the ImageReader.

   • onImageAvailable will give you back an android.media.Image and you can directly use FirebaseVisionImage.fromMediaImage(...) to create a FirebaseVisionImage. (You can find how to compute the rotation info from official doc here)

3. If you must do two Activities, then you need to work around the fact that android.media.Image is not Parcelable. I'd suggest you convert it to Bitmap first which is Parcelable and you can directly set it as an Intent extra (Up to you. Just thinking from end user's perspective, it's non-common to see the barcode being saved to my image gallery. So you might want to consider skipping the step of saving it to file). Later, in your 2nd Activity, you can use FirebaseVisionImage.fromBitmap(...).