algorithmstatisticsnlpnamed-entity-recognition

Methods for Geotagging or Geolabelling Text Content


What are some good algorithms for automatically labeling text with the city / region or origin? That is, if a blog is about New York, how can I tell programatically. Are there packages / papers that claim to do this with any degree of certainty?

I have looked at some tfidf based approaches, proper noun intersections, but so far, no spectacular successes, and I'd appreciate ideas!

The more general question is about assigning texts to topics, given some list of topics.

Simple / naive approaches preferred to full on Bayesian approaches, but I'm open.


Solution

  • You're looking for a named entity recognition system, or short NER. There are several good toolkits available to help you out. LingPipe in particular has a very decent tutorial. CAGEclass seems to be oriented around NER on geographical place names, but I haven't used it yet.

    If you're going with Java, I'd recommend using the LingPipe NER classes. OpenNLP also has some, but the former has a better documentation.

    If you're looking for some theoretical background, Chavez et al. (2005) have constructed an interesting system and documented it.