Convert CameraX Captured ImageProxy to Bitmap

Question

I was working with CameraX and had hard time converting captured ImageProxy to Bitmap. After searching and trying, I formulated a solution. Later I found that it was not optimum so I changed the design. That forced me to drop hours of work.

Since I (or someone else) might need it in a future, I decided to post here as a question and post and answer to it for reference and scrutiny. Feel free to add better answer if you have one.

The relevant code is:

class ImagePickerActivity : AppCompatActivity() {
    private var width = 325
    private var height = 205

    @RequiresApi(Build.VERSION_CODES.LOLLIPOP)
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_image_picker)

        view_finder.post { startCamera() }
    }

    @RequiresApi(Build.VERSION_CODES.LOLLIPOP)
    private fun startCamera() {
        // Create configuration object for the viewfinder use case
        val previewConfig = PreviewConfig.Builder().apply {
            setTargetAspectRatio(Rational(1, 1))
            //setTargetResolution(Size(width, height))
            setLensFacing(CameraX.LensFacing.BACK)
            setTargetAspectRatio(Rational(width, height))
        }.build()

        }

        // Create configuration object for the image capture use case
        val imageCaptureConfig = ImageCaptureConfig.Builder()
            .apply {
                setTargetAspectRatio(Rational(1, 1))
                // We don't set a resolution for image capture instead, we
                // select a capture mode which will infer the appropriate
                // resolution based on aspect ration and requested mode
                setCaptureMode(ImageCapture.CaptureMode.MIN_LATENCY)
            }.build()

        // Build the image capture use case and attach button click listener
        val imageCapture = ImageCapture(imageCaptureConfig)
        capture_button.setOnClickListener {

            imageCapture.takePicture(object : ImageCapture.OnImageCapturedListener() {
                override fun onCaptureSuccess(image: ImageProxy?, rotationDegrees: Int) {
                        //How do I get the bitmap here?
                        //imageView.setImageBitmap(someBitmap)
                }

                override fun onError(useCaseError: ImageCapture.UseCaseError?, message: String?, cause: Throwable?) {
                    val msg = "Photo capture failed: $message"
                    Toast.makeText(baseContext, msg, Toast.LENGTH_SHORT).show()
                    Log.e(localClassName, msg)
                    cause?.printStackTrace()
                }
            })
        }
        CameraX.bindToLifecycle(this, preview, imageCapture)
    }

}

Answer 1

Slightly modified version. Using the inline function use on the Closable ImageProxy

imageCapture.takePicture(
           object : ImageCapture.OnImageCapturedListener() {
               override fun onCaptureSuccess(image: ImageProxy?, rotationDegrees: Int) {
                     image.use { image ->
                           val bitmap: Bitmap? = image?.let {
                                imageProxyToBitmap(it)
                            } ?: return
                      }
          }
       })

  private fun imageProxyToBitmap(image: ImageProxy): Bitmap {
        val buffer: ByteBuffer = image.planes[0].buffer
        val bytes = ByteArray(buffer.remaining())
        buffer.get(bytes)
        return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
    } 

Answer 2

So the solution was to add extension method to Image and here is the code

class ImagePickerActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_image_picker)
    }

    private fun startCamera() {

        val imageCapture = ImageCapture(imageCaptureConfig)
        capture_button.setOnClickListener {

            imageCapture.takePicture(object : ImageCapture.OnImageCapturedListener() {
                override fun onCaptureSuccess(image: ImageProxy?, rotationDegrees: Int) {
                    imageView.setImageBitmap(image.image?.toBitmap())
                }
                //.....
            })
        }
    }

}

fun Image.toBitmap(): Bitmap {
    val buffer = planes[0].buffer
    buffer.rewind()
    val bytes = ByteArray(buffer.capacity())
    buffer.get(bytes)
    return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}

Answer 3

Here is the safest approach, using MLKit's own implementation. Tested and working on MLKit version 1.0.1

import com.google.mlkit.vision.common.internal.ImageConvertUtils;

Image mediaImage = imageProxy.getImage();
InputImage image = InputImage.fromMediaImage(mediaImage, imageProxy.getImageInfo().getRotationDegrees());
Bitmap bitmap = ImageConvertUtils.getInstance().getUpRightBitmap(image)

Answer 4

Java Implementation of Backbelt's Answer.

private Bitmap imageProxyToBitmap(ImageProxy image) {
    ByteBuffer buffer = image.getPlanes()[0].getBuffer();
    byte[] bytes = new byte[buffer.remaining()];
    buffer.get(bytes);
    return BitmapFactory.decodeByteArray(bytes,0,bytes.length,null);
}