capture overlay using PreviewView of cameraX

Question

I'm trying to capture a picture with overlay included in image capture. I was able to set overlay to previewView using cameraView.overlay.add(binding.textView). How ever, it did not save when trying to save an image with imageCapture Only the picture was saved not the overlay. How do I save an image with overlay included using PreviewView of camera x.

Please don't mark this as duplicate. I researched a lot and most of the example online are using the old camera api which does not apply to camera x library. Any help is appreciated. Thanks in advance.

Here is my code

           <FrameLayout
                android:id="@+id/camera_wrapper"
                android:layout_width="match_parent"
                android:layout_height="0dp"
                app:layout_constraintTop_toTopOf="@id/space1"
                app:layout_constraintBottom_toBottomOf="@id/space">
    
                <androidx.camera.view.PreviewView
                    android:id="@+id/camera_view"
                    android:layout_width="match_parent"
                    android:layout_height="match_parent" />
    
                <TextView
                    android:id="@+id/text_view"
                    android:layout_width="wrap_content"
                    android:layout_height="wrap_content"
                    android:layout_gravity="center"
                    android:text="Hello world"
                    android:textSize="42sp"
                    android:textColor="@android:color/holo_green_dark"/>
    
            </FrameLayout>

  private lateinit var outputDirectory: File
  private lateinit var cameraExecutor: ExecutorService
  private var preview: Preview? = null
  private var lensFacing: Int = CameraSelector.LENS_FACING_FRONT
  private var imageCapture: ImageCapture? = null
  private var camera: Camera? = null
  private var cameraProvider: ProcessCameraProvider? = null

override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
    super.onViewCreated(view, savedInstanceState)
    outputDirectory = getOutputDirectory()
    cameraExecutor = Executors.newSingleThreadExecutor()
  }

  private fun setupCamera() {
    val cameraProviderFuture = ProcessCameraProvider.getInstance(requireContext())

    cameraProviderFuture.addListener(
      Runnable {
        // Used to bind the lifecycle of cameras to the lifecycle owner
        cameraProvider = cameraProviderFuture.get()

        // Get screen metrics used to setup camera for full screen resolution
        val metrics = DisplayMetrics().also { binding.cameraView.display.getRealMetrics(it) }
        Timber.d("Screen metrics: ${metrics.widthPixels} x ${metrics.heightPixels}")

        val screenAspectRatio = aspectRatio(metrics.widthPixels, metrics.heightPixels)
        Timber.d("Preview aspect ratio: $screenAspectRatio")

        val rotation = binding.cameraView.display.rotation

        // CameraProvider
        val cameraProvider = cameraProvider
          ?: throw IllegalStateException("Camera initialization failed.")

        // CameraSelector
        val cameraSelector = CameraSelector.Builder().requireLensFacing(lensFacing).build()

        // add text overlay *---------*
        binding.cameraView.overlay.add(binding.textView)

        // Preview
        preview = Preview.Builder()
          // We request aspect ratio but no resolution
          .setTargetAspectRatio(screenAspectRatio)
          // Set initial target rotation
          .setTargetRotation(rotation)
          .build()

        // ImageCapture
        imageCapture = ImageCapture.Builder()
          .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
          // We request aspect ratio but no resolution to match preview config, but letting
          // CameraX optimize for whatever specific resolution best fits our use cases
          .setTargetAspectRatio(screenAspectRatio)
          // Set initial target rotation, we will have to call this again if rotation changes
          // during the lifecycle of this use case
          .setTargetRotation(rotation)
          .build()

        // Must unbind the use-cases before rebinding them
        cameraProvider.unbindAll()

        try {
          // A variable number of use-cases can be passed here -
          // camera provides access to CameraControl & CameraInfo
          camera = cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageCapture)
          // Attach the viewfinder's surface provider to preview use case
          preview?.setSurfaceProvider(binding.cameraView.surfaceProvider)
        } catch (exc: Exception) {
          Toast.makeText(requireContext(), "Something went wrong. Please try again.", Toast.LENGTH_SHORT).show()
          findNavController().navigateUp()
        }
      },
      ContextCompat.getMainExecutor(requireContext())
    )
  }

private fun takePhoto() {
    imageCapture?.let { imageCapture ->

      // Create output file to hold the image
      val photoFile = createFile(outputDirectory, FILENAME, PHOTO_EXTENSION)

      // Setup image capture metadata
      val metadata = ImageCapture.Metadata().apply {

        // Mirror image when using the front camera
        isReversedHorizontal = lensFacing == CameraSelector.LENS_FACING_FRONT
      }

      // Create output options object which contains file + metadata
      val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile)
          .setMetadata(metadata)
          .build()

      // Setup image capture listener which is triggered after photo has been taken
      imageCapture.takePicture(outputOptions, cameraExecutor, object : ImageCapture.OnImageSavedCallback {
        override fun onError(exc: ImageCaptureException) {
          Timber.e(exc, "Photo capture failed: ${exc.message}")
        }

        override fun onImageSaved(output: ImageCapture.OutputFileResults) {
          val savedUri = output.savedUri ?: Uri.fromFile(photoFile)
          Timber.d("Photo capture succeeded: $savedUri")

          // Implicit broadcasts will be ignored for devices running API level >= 24
          // so if you only target API level 24+ you can remove this statement
          if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {
            requireActivity()
                .sendBroadcast(Intent(android.hardware.Camera.ACTION_NEW_PICTURE, savedUri))
          }


          // If the folder selected is an external media directory, this is
          // unnecessary but otherwise other apps will not be able to access our
          // images unless we scan them using [MediaScannerConnection]
          val mimeType = MimeTypeMap.getSingleton()
              .getMimeTypeFromExtension(savedUri.toFile().extension)
          MediaScannerConnection.scanFile(
              context,
              arrayOf(savedUri.toFile().absolutePath),
              arrayOf(mimeType)
          ) { _, uri ->
            Timber.d("Image capture scanned into media store: $uri")
          }
        }
      })
    }

  }

Answer 1

You must overlay the text over the image yourself. I would suggest to use takePicture(Executor, …) that puts the Jpeg in memory; then, overlay your text using one of the libraries (not part of Android framework, neither of Jetpack), and save the result in file.

If you can compromise on image quality, you can draw the Jpeg on Bitmap canvas, and draw your text on top.