Reading / converting camera stream images in Flutter

Question

On my iOS device, I'm accessing the camera through a CameraController (provided by the camera package) with controller.startImageStream((CameraImage img) {...}

The data coming out of the camera is in a bgra8888 format on my phone, but I've read that it's in yuv420 on android devices. To convert the image stream data to a usable, consistent format, I'm using:

import 'dart:typed_data';
import 'package:camera/camera.dart';
import 'package:image/image.dart' as im;

Uint8List concatenatePlanes(List<Plane> planes) {
  final WriteBuffer allBytes = WriteBuffer();
  planes.forEach((Plane plane) => allBytes.putUint8List(plane.bytes));
  return allBytes.done().buffer.asUint8List();
}

List<CameraDescription> cameras = await availableCameras();
CameraController controller = CameraController(cameras[0], ResolutionPreset.medium);

controller.startImageStream((CameraImage img) {
  print(img.format.group); // returns ImageFormatGroup.bgra8888

  List<int> imgData = concatenatePlanes(img.planes);
  im.Image image = im.decodeImage(imgData);
});

The imgData variable is full of data streaming off the camera, but the converted image returned from decodeImage is null.

I had read on other posts that the image package would be up to the task of decoding bgra8888/yuv420 images (https://stackoverflow.com/a/57635827/479947) but I'm not seeing support in its formats.dart source (https://github.com/brendan-duncan/image/blob/master/lib/src/formats/formats.dart).

The target Image format is defined as:

An image buffer where pixels are encoded into 32-bit unsigned ints (Uint32). Pixels are stored in 32-bit unsigned integers in #AARRGGBB format. This is to be consistent with the Flutter image data.

How would I get my image stream image in bgra8888/yuv420 converted into the desired Image format?

Answer 1

Have you check this? convertImage

If you convert from bgra8888 to Image, it's kinda fast but yuv420 to Image take much more time, so if you have problem about performance, I can help a little bit with my experience. By the way, if you have performance problem and trying using Isolate, u will meet the memory issue.