redisfull-text-searchfull-text-indexingredisearch

RediSearch - searching for particular word which occurs in many records take long time. How to improve it?


I have addresses database (as hashes) with about 30 millions records. I was adding text index to all addresses fields. Searching looks ok until I want to search word which occur in many records. For example searchin word "London" which occur in about 2500000 records took 4,5 seconds (FT.SEARCH idx:a4 london LIMIT 0 2). Is it any possibility to improve this result, any changes to make? Thank you for help.


Solution

  • If you do not care about getting the first 2 results sorted by scoring (calculated by tfidf), you can use FT.AGGREGATE which will just return after finding the first 2 results (without getting all the results, calculating the score, sort them, and get the first 2 results). It should look like this:

    FT.AGGREGATE idx:a4 london LIMIT 0 2
    

    Notice that you should use LOAD to decide which fields to return from the hash. Please refer here for the full FT.AGGREGATE documentation: https://oss.redislabs.com/redisearch/Aggregations/

    Again if you chose to use it, know that you are losing sorting by results score.