image-processingcomputer-visionartificial-intelligenceicr

Detect handwritten characters in boxes from a filled form using Fourier transforms


I am trying to extract handwritten characters from boxes. The scanning of the forms is not consistent, so the width and height of the boxes are also not constants.

Here is a part of the form.

enter image description here

My current approach:
1. Extract horizontal lines
2. Extract vertical lines
3. Combine both the above images
4. Find contours ( used opencv)

This approach gives me most of the boxes. But, when the box is filled with characters like "L" or "I", the vertical line in the character is also getting extracted as a part of vertical lines extraction. Hence the contours also get messed up.
Since the boxes are arranged periodically, is there a way to extract the boxes using Fast Fourier transforms?


Solution

  • I recently came up with a python package that deals with this exact problem.
    I called it BoxDetect and after installing it through:

    pip install boxdetect
    

    It may look somewhat like this (you need to adjust parameters for different forms:

    from boxdetect import config
    
    config.min_w, config.max_w = (20,50)
    config.min_h, config.max_h = (20,50)
    config.scaling_factors = [0.4]
    config.dilation_iterations = 0
    config.wh_ratio_range = (0.5, 2.0)
    config.group_size_range = (1, 100)
    config.horizontal_max_distance_multiplier = 2
    
    
    from boxdetect.pipelines import get_boxes
    
    image_path = "dumpster/m1nda.jpg"
    rects, grouped_rects, org_image, output_image = get_boxes(image_path, config, plot=False)
    

    enter image description here

    You might want to check below thread for more info:
    How to detect all boxes for inputting letters in forms for a particular field?