elasticsearchelasticsearch-aggregationelasticsearch-dslelasticsearch-query

Elasticsearch Query to Retrieve a Fixed Number of Documents from Each Index


I have an Elasticsearch setup with multiple indexes (let's say a, b, c, d, e, and f). My goal is to retrieve a total of 20 documents, with exactly 4 documents from each index. However, if any index has fewer than 4 documents, I want to fill in the remaining documents from other indexes.

For example:

a = 4
b = 4
c = 4
d = 4
e = 4
Total = 20

If any index has less than 4 documents:

a = 3
b = 4 or 5
c = 4 or 5
d = 4 or 5
e = 4 or 5
Total = 20

What should be the Elasticsearch query or payload to achieve this?

Current approach:

{
    "index": [
        "_all"
    ],
    "body": {
        "query": {
            "match_all": {}
        },
        "size": 20,
        "sort": {
            "created_at": "desc"
        }
    }
}

Any help or guidance is appreciated.

Thanks in advance!


Solution

  • I would probably use the multi-search API to do this and query each index separately, like this:

    POST /_msearch
    {"index": "a"}
    {"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
    {"index": "b"}
    {"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
    {"index": "c"}
    {"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
    {"index": "d"}
    {"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
    {"index": "e"}
    {"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
    

    Also note that I'm retrieving more than 4 hits per index, for the case where an index would have less than 4 hits, you can pick another hit from another index.