android-camera2firebase-mlkittext-recognitionandroid-graphicscamera-api

How to display text detected from ML Kit and display it on the actual location of the word


I am working on an OCR app for android and am using Camera2Api.I am using android-camera2basic as a boiler plate code and i used ml kit for text recognition.

Am facing a strange problem the GraphicOverlay Is not scaling properly its only covering half screen.The GraphicOverlay is not drawing properly on the detected words.

Click On this To View The Problem

As you can see that the graphic overlay is not drawing where it should for example "Stack Exchange Network" graphic overaly is not displaying it on the top of Stack Exchange Network.

Here are my Files


Solution

  • The thing is that the library detect words on the original bitmap that you've passed (with its original width and height), and then returns the location of the words according to that image. But the image shown on the screen is a SCALED image with a new width and height to fit the screen or the imageView. So what you wanna do is, rescaling the bitmap according to your imageView and then pass the new bitmap

    //get the scaled bitmap image with the new width and height showed on the screen
        private Bitmap getScaledBitmap (Bitmap bitmapImage){
    
            //width and height of original image
            final int imageWidth = bitmapImage.getWidth();
            final int imageHeight = bitmapImage.getHeight();
    
            //width and height of the imageView
            final int imageViewWidth  = mImageView.getMeasuredWidth();
            final int imageViewHeight = mImageView.getMeasuredHeight();
    
            final int scaledWidth , scaledHeight;
    
    
            if (imageWidth*imageViewHeight <= imageHeight*imageViewWidth) {
    
                //rescaled width and height of image within ImageView
                scaledWidth = (imageWidth*imageViewHeight)/imageHeight;
                scaledHeight = imageViewHeight;
            }
            else {
                //rescaled width and height of image within ImageView
                scaledWidth = imageViewWidth;
                scaledHeight = (imageHeight*imageViewWidth)/imageWidth;
            }
    
    
            return Bitmap.createScaledBitmap(bitmapImage, scaledWidth, scaledHeight, true);
        }
    

    I used this answer for scaling the image: https://stackoverflow.com/a/13318469/9242141


    Edit: Sorry, I noticed that you're not using Bitmaps in your code and things could be a little different in your case but I think you can find a solution based on this idea as I have faced the same problem as you and this worked perfectly for me.