machine-learningneural-networknltkstreet-addresstext-classification

How to check an input string contains street address or not?


We want to identify the address fields from a document. For Identifying the address fields we converted the document to OCR files using Tesseract. From the tesseract output we want to check a string contains the address field or not . Which is the right strategy to resolve this problem ?

  1. Its not possible to solve this problem using the regex because address fields are different for various documents and countries
  2. Tried NLTK for classifying the words but not works perfectly for address field.

Required output

I am staying at 234 23 Philadelphia - Contains address files <234 23 Philadelphia>

I am looking for a place to stay - Not contains address 

Provide your suggestions to solve this problem .


Solution

  • As in many ML problems, there are mutiple posible solutions, and the important part(and the one commonly has greater impact) is not which algorithm or model you use, but feature engineering ,data preprocessing and standarization ,and things like that. The first solution comes to my mind(and its just an idea, i would test it and see how it performs) its:

    1. Get your training set examples and list the "N" most commonly used words in all examples(thats your vocabulary), this list will contain every one of the "N" most used words , every word would be represented by a number(the list index)
    2. Transform your training examples: read every training example and change its representation replacing every word by the number of the word in the vocabolary.
    3. Finally, for every training example create a feature vector of the same size as the vocabulary, and for every word in the vocabulary your feature vector will be 0(the corresponding word doesnt exists in your example) or 1(it exists) , or the count of how many times the word appears(again ,this is feature engineering)
    4. Train multiple classifiers ,varing algorithms,parameters, training set sizes, etc, and do cross validation to choose your best model.

    And from there keep the standard ML workflow...