[SOLVED] Using ZXing to analyze camera preview on Android

Using ZXing to analyze camera preview on Android

I want to write a maui app where I use the ZXing BarcodeReaderGeneric to analyzer the camera preview from the Camera2 of my android phone.

Assuming the camera is already running, I am using the Android.Media.ImageReader to get notified when a new image is available from the camera.

m_ImageReader = ImageReader.NewInstance(camera.Width, camera.Height, ImageFormatType.Yuv420888, 2);
m_ImageReader.SetOnImageAvailableListener

When an image is available, I want to use the barcode reader to decode a barcode on that image as fast as possible.

At first, I tried to analyze only the first plane of the image by using an RGBLuminanceSource to analyze the whole image, which worked but turned out to be very slow.

Then I read an PlanarYUVLuminanceSource would be the better options for the camera image. So I took all three planes of the image and put them in a single buffer, like

ByteBuffer yBuffer = image.GetPlanes()[0].Buffer;
ByteBuffer uBuffer = image.GetPlanes()[1].Buffer;
ByteBuffer vBuffer = image.GetPlanes()[2].Buffer;

int yCount = yBuffer.Remaining();
int uCount = uBuffer.Remaining();
int vCount = vBuffer.Remaining();

byte[] yBytes = new byte[yCount];
byte[] uBytes = new byte[uCount];
byte[] vBytes = new byte[vCount];

yBuffer.Get(yBytes, 0, yCount);
uBuffer.Get(uBytes, 0, uCount);
vBuffer.Get(vBytes, 0, vCount);

byte[] allBytes = new byte[yCount + uCount + vCount];

System.Buffer.BlockCopy(yBytes, 0, allBytes, 0, yCount);
System.Buffer.BlockCopy(uBytes, 0, allBytes, yCount, uCount);
System.Buffer.BlockCopy(vBytes, 0, allBytes, yCount + uCount, vCount);

but when I use the PlanarYUVLuminanceSource like

var source = new PlanarYUVLuminanceSource(allBytes, image.Width, image.Height, 0, 0, image.Width, image.Height, false);
var result = m_BarcodeReader.Decode(source);

the barcode is never found and result is always null. I don't know why.

After this I wanted to try the cropping feature of the PlanarYUVLuminanceSource to not have to decode the whole image, but even the decoding of the whole image is not working. Maybe one more idea is to make a 640x480 image from the camera image, but I don't have a clue how to do that.

So can anyone tell me what the best practice workflow is in such a situation?

Solution

I'd recommend looking at newer barcode libraries like https://developers.google.com/ml-kit/vision/barcode-scanning. ZXing was designed to use the old camera API, which produces YUV data in a different format than the newer camera2 API does.

If you do want to use ZXing, you'll need to translate ImageFormat.YUV_420_888 into ImageFormat.NV21. Mainly, this means handling the row stride correctly.