Transforming CameraImage to bitmap in Flutter

We are currently capturing camera images in Flutter as CameraImages in the YUV_420_888 format. Our goal is to convert these images into a format that can be used with BitmapFactory in Kotlin to process the images, extract landmarks using MediaPipe, and perform recognition on them. We are connecting Flutter with Kotlin via Platform Channels.

However, we are encountering issues. When we try to convert the received array into a bitmap in Kotlin, we are getting null values. Additionally, converting these image types to RGB in Flutter and then passing them to arrays has proven to be costly and potentially affects performance.

If anyone has alternative solutions or methods to effectively process images captured from Flutter's camera using MediaPipe or other modules, your suggestions would be greatly appreciated.

Thank you!

We tried converting image to array and then passing it to Kotlin -> didn't work. We also tried to transform images from CameaImage to RGB (although we knew it had a bad performance) and didn't work too.

Did you use the image package? If not, you can convert the cameraImages to an Image package image and then convert that to any format.

use this to convert the cameraImage ->img_lib.Image

import 'package:image/image.dart' as image_lib;
image_lib.Image convertYUV420ToImage(CameraImage cameraImage) {
  final width = cameraImage.width;
  final height = cameraImage.height;

  final uvRowStride = cameraImage.planes[1].bytesPerRow;
  final uvPixelStride = cameraImage.planes[1].bytesPerPixel!;

  final yPlane = cameraImage.planes[0].bytes;
  final uPlane = cameraImage.planes[1].bytes;
  final vPlane = cameraImage.planes[2].bytes;

  final image = image_lib.Image(width: width, height: height);

  var uvIndex = 0;

  for (var y = 0; y < height; y++) {
    var pY = y * width;
    var pUV = uvIndex;

    for (var x = 0; x < width; x++) {
      final yValue = yPlane[pY];
      final uValue = uPlane[pUV];
      final vValue = vPlane[pUV];

      final r = yValue + 1.402 * (vValue - 128);
      final g = yValue - 0.344136 * (uValue - 128) - 0.714136 * (vValue - 128);
      final b = yValue + 1.772 * (uValue - 128);

      image.setPixelRgba(x, y, r.toInt(), g.toInt(), b.toInt(), 255);

      pY++;
      if (x % 2 == 1 && uvPixelStride == 2) {
        pUV += uvPixelStride;
      } else if (x % 2 == 1 && uvPixelStride == 1) {
        pUV++;
      }
    }

    if (y % 2 == 1) {
      uvIndex += uvRowStride;
    }
  }
  return image;
}

Then you can use to encode it to any format like bmp;

final bmpImage = image_lib.encodeBmp(image);

Image package use native code to encode the images.