opensearchamazon-opensearchelasticsearch-opendistro

Is it possible to define an ISM/ILM policy such that an action is only performed when all the conditions are met?


For instance, I want to rollover my index only when both index_age:"1h" AND doc_count:1

{
"policy": {
    "description": "Example rollover policy.",
    "default_state": "rollover",
    "states": [
        {
            "name": "rollover",
            "actions": [
                {
                    "rollover": {
                        "min_index_age": "1h",
                        "min_doc_count": 1
                    }
                }
            ],
            "transitions": []
        }
    ],
    "ism_template": {
        "index_patterns": [
            "log*"
        ],
        "priority": 100
    }
}

Doing at GET /_opendistro/_ism/explain/log-000001?pretty gave (excluded few fields):

{
"info": {
  "message": "Successfully rolled over index [index=log-000001]",
  "conditions": {
    "min_index_age": {
      "condition": "1h",
      "current": "1.1h",
      "creationDate": 1685093175627
    },
    "min_doc_count": {
      "condition": 1,
      "current": 0
    }
  }
}

Even though the 2nd condition was not met, still the rollover happened.


Solution

  • OPENSEARCH

    Currently, there are only min conditions, and whenever one of the conditions is satisfied it will trigger the rollover.

    Rollover Official documentation

    Rolls an alias over to a new index when the managed index meets one of the rollover conditions.

    ELASTICSEARCH

    Rollover official documentation

    There are max and min conditions available for Elasticsearch. You can use multiple max and min conditions.

    The index will rollover if any max_* condition is satisfied and all min_* conditions are satisfied.

    An example:

    The following request only rolls over the index if the current write index meets one or more of the following conditions:


    POST my-data-stream/_rollover
    {
      "conditions": {
        "max_age": "7d",
        "max_docs": 1000,
        "max_primary_shard_size": "50gb",
        "max_primary_shard_docs": "2000"
      }
    }
    

    NOTE: max conditions released on Elasticsearch version 8.4 and 7.17.6, it's available for the versions I mentioned and above.