How to segment handwritten digits

I am working on a problem of handwritten digit recognition. Basically, we need to recognize certain fields in documents like amount, account number, mobile number, etc.

Handwritten Digit recognition can be divided into 2 steps

1. Digit segmentation
2. Recognition of segmented digits

For step-2 we can use some pertained MNIST models but the problem is how to segment the digits. I tried OpenCV contours but it is only helpful when digits are separated with blank pixels i.e. when they do not touch each other, but often user writes numbers with touching or connected digits

Can anyone suggest some deep learning or non deep learning based methods for this task

Sample Images

For deep learning based approach, you can use mask rcnn. It is a very powerful approach and can be used to detect, localize and segment. The algorithm is powerful enough to recognize different classes even if they are close together.

It will generate bounding boxes around the digits and classify them.

Please look into this repository as my explanation would not do justice to the same.

It also contains sample examples for you to learn on. The only thing which may slow you down is that you will have to annotate your images. But using transfer learning you can reduce the amount of data you would actually use to train.

Here are some more relevant links:

Training MaskRCNN on your own dataset

Implementation on your own dataset

To understand more about MaskRCNN

Transfer Learning

There will of course be better articles on the above topics, but these are the ones I've used. Hopefully someone else will suggest them to you.