javascripthtmlocrgoogle-vision

Reading from Vision API Text Detection and Filling Appropriate fields


Have been trying to read data out of an Govt. Issued identity card and fill the fields of the form like following using google's Vision Api..

Government ID

I've successfully read the data from the vision API but now facing problems filling the form like following with appropriate data..

Form

How can i achieve this?

What i Wish to get

The response from Vision API:

{
"responses": [
    {
        "textAnnotations": [
            {
                "locale": "en",
                "description": "amagas faATST\nINCOME TAX DEPARTMENT\nMAHENDRAKUMARRBAGUL\nRAMKRISHNA NATTHU BAGUL\n01/06/1981\n4Permanent Account Number\nANSAB4834E\nSignature\nGOVT OF INDIA\n",
                "boundingPoly": {
                    "vertices": [
                        {
                            "x": 2,
                            "y": 64
                        },
                        {
                            "x": 4308,
                            "y": 64
                        },
                        {
                            "x": 4308,
                            "y": 2701
                        },
                        {
                            "x": 2,
                            "y": 2701
                        }
                    ]
                }
            },
            {
                "description": "amagas",
                "boundingPoly": {
                    "vertices": [
                        {
                            "x": 6,
                            "y": 64
                        },
                        {
                            "x": 774,
                            "y": 65
                        },
                        {
                            "x": 774,
                            "y": 374
                        },
                        {
                            "x": 6,
                            "y": 373
                        }
                    ]
                }
            },

Kindly Help


Solution

  • You can do this using Node.js. I did it using Node.js using the Microsoft's Computer Vision API. After you get the JSON string, parse that into a JSON object and run a loop to extract data from it. After that use the split function to get the data stored into arrays.

    //Load the request module
    var request = require('request');
    
    var str="";
    //Lets configure and request
    request({
        url: 'https://api.projectoxford.ai/vision/v1.0/ocr?', //URL to hit
        qs: {"language": "unk",
             "detectOrientation ": "true"
             }, //Query string data
    
        method: 'POST', //Specify the method
    
        headers: { //We can define headers too
            'Content-Type': 'application/json',
            'Ocp-Apim-Subscription-Key':'xxxxxxxxxxxxxxxx'
        },
    
        body: "{'url':'LINK TO THE IMAGE'}",
    
    }, function(error, response, body){
        if(error) {
            console.log(error);
        } else {
            var jsonObj = JSON.parse(body);
    
            var ob = jsonObj;
                for(i=0;i<ob.regions.length;i++){
                    for(j=0;j<ob.regions[i].lines.length;j++){
                        for(k=0;k<ob.regions[i].lines[j].words.length;k++){
                                var str = str + " "+ob.regions[i].lines[j].words[k].text;
                        }
                        str = str + "\n";
                    }
                }
    
    
                var arr = str.split("\n");
    
                console.log("Name: " + arr[1]);
                console.log("Father's Name: " + arr[2]);
                console.log("Date of Birth: " + arr[3]);
                console.log("Permanent Account Number: " + arr[5]);
    
        }
    });
    

    Just use your own Microsoft Computer Vision API Subscription Key in this. If you want to use your own JSON file generated from your Google Vision API, just scratch off the upper code and use the algorithm at the lower part of the code. It will work! :) Cheers