ImageAnalyzer ML Kit bounding box mislined

Question

I've got a simple layout:

 <RelativeLayout
    android:id="@+id/myP"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    >

    <androidx.camera.view.PreviewView
        android:id="@+id/mPreviewView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"/>

    <com.example.dochjavatestimplementation.pkgActivity.ExtendedImageView
        android:id="@+id/imageViewOmgShowIt"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:visibility="visible" />

</RelativeLayout>

The previewView is used to display the camera, the ExtendedImageView is used to display the display the found object rect.

My ImageAnalyzer:

imageAnalysis = new ImageAnalysis.Builder()
 .setTargetResolution(new Size(mPreviewView.getWidth(),mPreviewView.getHeight()))
 .build();

imageAnalysis.setAnalyzer(executor, new PaperImageAnalyser());

PaperImageAnalyser:

public class PaperImageAnalyser implements ImageAnalysis.Analyzer {


ObjectDetectorOptions options = new ObjectDetectorOptions.Builder()
    .setDetectorMode(ObjectDetectorOptions.STREAM_MODE)
    .enableClassification()  // Optional
    .build();

ObjectDetector objectDetector = ObjectDetection.getClient(options);



@Override
public void analyze(@NonNull ImageProxy imageProxy) {
@SuppressLint("UnsafeExperimentalUsageError") Image mediaImage = imageProxy.getImage();

if (mediaImage != null) {
    InputImage image =InputImage.fromMediaImage(mediaImage, imageProxy.getImageInfo().getRotationDegrees());

    objectDetector.process(image)
        .addOnSuccessListener(
            new OnSuccessListener<List<DetectedObject>>() {
                @Override
                public void onSuccess(List<DetectedObject> detectedObjects) {
        
                    for (DetectedObject detectedObject : detectedObjects) {
                        Rect boundingBox = detectedObject.getBoundingBox();
                        CameraImp.imageViewOmgShowIt.drawFoundObj(boundingBox);
                    }

                }
            })
        .addOnFailureListener(
            new OnFailureListener() {
                @Override
                public void onFailure(@NonNull Exception e) {
                    Log.e("error",""+e.getMessage());
                }
            })
        .addOnCompleteListener(new OnCompleteListener<List<DetectedObject>>() {
            @SuppressLint("UnsafeExperimentalUsageError")
            @Override
            public void onComplete(@NonNull Task<List<DetectedObject>> task) {
                imageProxy.getImage().close();
                imageProxy.close(); 

            }
        });
    }
  }
}

Drawing the rect:

public void drawFoundObj(Rect boundingBox) {
    foundYo = boundingBox;
    invalidate(); //call draw
}

and

Paint mPaint = new Paint();
Canvas mCanvas = new Canvas();
@Override
protected void onDraw(Canvas canvas) {

    //clear exsisting rects 
    Xfermode x = mPaint.getXfermode();
    mPaint.setXfermode(new PorterDuffXfermode(PorterDuff.Mode.CLEAR));
    mPaint.setXfermode(x);
    mCanvas.drawPaint(mPaint);


    //set current rect
    Paint paint = new Paint();
    paint.setColor(Color.YELLOW);
    paint.setStrokeWidth(3);

    paint.setAlpha(40);
    canvas.drawRect(foundYo,paint);
}

The results look like this:

As you can see the issue is that although the object gets somehow recognizes, the coordinates are way wrong compared to the position of the object.

What might the cause be?

Is the resolution of the image I receive from the imageanalyzer smaller so that causes maybe that the coordinates are smaller thus wronger?

Or am I drawing the image wrong?

Answer 1

You will need to get the width and height of the analyzed image and transform the boundingbox to the right size for display. Please refer to this example from mlkit vision_quickstart as a reference. https://github.com/googlesamples/mlkit/blob/74d5edb101d1e2fb8bd404c41a684b71a06d507a/android/vision-quickstart/app/src/main/java/com/google/mlkit/vision/demo/java/CameraXLivePreviewActivity.java#L421