vespaapproximate-nn-searching

How filtered document list look up work in nearest neighbour search prefiltering


In pre-filter based ANN, once we have list of documents after applying pre-filter, vespa starts hsnw algorithm to find nearest neighbours. In hsnw algorithm, vespa starts with a node and look for the neighbours which are present in pre-filter list, How search of neighbour in document list is implemented in vespa ? it's linear search or hashing ?


Solution

  • This blog post has an excellent overview of how Vespa combines filters with HNSW search https://blog.vespa.ai/constrained-approximate-nearest-neighbor-search/.

    In pre-filter case, the resulting "allow" document list is a bitvector, and the lookup is O(1).