elasticsearchmergeelasticsearch-indices

How to merge elasticsearch indices


In Elasticsearch v7.12.0 I have several indices named as payload.YYYY-MM. With YYYY = year and MM = month.

In order to reduce the amount of indices, I would like to merge all the indices that are from the same year in a new indice named payload.YYYY

How can I do it using the Elasticsearch API ?

(I don't want to use the aliases as it doesn't reduce the amount of indices)


Solution

  • You can do this very simply using the _reindex API and running the following command for each year you want to support:

    POST _reindex?wait_for_completion=false
    {
      "source": {
        "index": "payload.2022-*"
      },
      "dest": {
        "index": "payload.2022"
      }
    }
    

    All indexes named payload.2022-* will be reindexed into the yearly index payload.2022. Make sure to create the yearly index with the same mapping as the monthly indexes prior to running the command.

    Note: adding wait_for_completion=false makes the command create a task in the background and return immediately with a task ID. You can then use that task ID to monitor the process in the background using

    GET _tasks/<task_id>