Redis vector search does not return required number of documents for large K

I have deployed a RedisSearch server to test vector similarity search. I have followed this guide pretty much exactly, using HNSW algorithm:

Using Redis as a Vector Database with OpenAI

My main goal was to test RedisSearch performance for medium size datasets (~20 million documents) and large number of returned documents (~5_000 to 10_000) because ElasticSearch solution for vector similarity search was very slow once we start increasing k and num_candidates to return a large resulting set.

We have an embedding model and based on tests with ES it generates adequate embeddings.

So since we only needed to test the performance (time aspect) of Redis VSS, to save time and money on generating real embeddings with our model, I generated fake embeddings for every document by:

list(np.random.rand(1024))

and then indexed 20 million documents into Redis. So now when I perform search and request 1_000 documents I do get 1_000 documents back consistently. But requesting 5_000 documents or more, I start getting either empty sets or sets with number of documents less than specified amount.

This part I do not understand. Nowhere in the code I specified that I needed documents with similarity score above some specific value. I just need 5_000 or 10_000 documents with any score.

How can I have Redis VSS return required number of documents with any similarity score?

Search snippet:

def search_redis(
    redis_client: redis.Redis,
    user_query: str,
    index_name: str = INDEX_NAME,
    vector_field: str = "embedding",
    return_fields: list = ["id", "user_id", "title", "vector_score"],
    hybrid_fields = "*",
    k: int = 100,
):
    embedded_query = get_fake_embeddings() # 1024 random floats
    base_query = f'{hybrid_fields}=>[KNN {k} @{vector_field} $vector AS vector_score]'
    query = (
        Query(base_query)
         .return_fields(*return_fields)
         .sort_by("vector_score")
         .paging(0, k)
         .dialect(2)
    )
    params_dict = {"vector": np.array(embedded_query).astype(dtype=np.float32).tobytes()}
    results = redis_client.ft(index_name).search(query, params_dict)
    return results.docs

Solution

This is probably a result of the values for the TIMEOUT and ON_TIMEOUT configuration options, which I'm guessing you haven't touched.

By default, TIMEOUT it is 500ms. So, if your query gets long, it'll timeout after 500ms. Easy enough.

But, the ON_TIMEOUT option determines what happens upon timeout. And it defaults to RETURN. So, upon timeout, calls to FT.SEARCH will just return what it has found so far.

If you want to see if this is the problem, try setting ON_TIMEOUT to FAIL and see if it, well, fails. If so, you can disable timeouts by setting TIMEOUT to 0.

Details on these options and how to get and set these are here but a quick way to do this would be to use the FT.CONFIG command. Like this:

redis.cloud> FT.CONFIG GET ON_TIMEOUT
1) 1) ON_TIMEOUT
   2) "return"
redis.cloud> FT.CONFIG GET TIMEOUT
1) 1) TIMEOUT
   2) "500"

redis.cloud> FT.CONFIG SET ON_TIMEOUT FAIL
OK
redis.cloud> FT.CONFIG SET TIMEOUT 0
OK

redis.cloud> FT.CONFIG GET ON_TIMEOUT
1) 1) ON_TIMEOUT
   2) "fail"
redis.cloud> FT.CONFIG GET TIMEOUT
1) 1) TIMEOUT
   2) "0"