androidflutterimagekotlinevent-channel

Send Android image over event channel to Dart


In a Flutter plugin, I want to send an Android image over an Event channel to Dart and later convert it to a Dart image. The following experimental code works and can be seen as a starting point.

        val imageAnalysis = ImageAnalysis.Builder()
            // .setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)
            // Default is OUTPUT_IMAGE_FORMAT_YUV_420_888
            // .setTargetResolution(Size(1280, 720))
            .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
            .build()
        imageAnalysis.setAnalyzer(executor, ImageAnalysis.Analyzer { imageProxy ->
            val rotationDegrees = imageProxy.imageInfo.rotationDegrees
            val mediaImage = imageProxy.image
            if (mediaImage != null) {
                val image = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
                sink?.success(image.width)
                sink?.success(image.height)
                sink?.success(image.planes!![0].rowStride)
                sink?.success(image.planes!![0].pixelStride)
                sink?.success(image.planes!![0].buffer.asShortBuffer()[0])
            }
            imageProxy.close()
        })
:
cameraProvider.bindToLifecycle(owner, cameraSelector, preview, imageCapture, imageAnalysis)

What I really want to do is sink?.success(image) but that does not work - you cannot send an object over an Event channel.

Has anyone done this? Any idéas? Is there working code somewhere?


Solution

  • This works. I get the image in RGBA888 instead of YUV_420_888.

    val analyzer = ImageAnalysis.Analyzer { image ->
        if (image.format == PixelFormat.RGBA_8888) {
            val planes = image.image!!.planes
            val plane0 = planes!![0]
            val byteBuffer: ByteBuffer = plane0.buffer
            val byteArray = ByteArray(byteBuffer.capacity())
            byteBuffer.get(byteArray)
            sink?.success(byteArray)
            // Thread.sleep(500)
        }
        image.close()
    }
    val imageAnalysis = ImageAnalysis.Builder()
        .setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)
        .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
        .build()
    imageAnalysis.setAnalyzer(executor, analyzer)