elasticsearchmonitoringelasticsearch-indices

Check writes happening to an old Elasticsearch index


Now due to the disk space issue on the Elasticsearch cluster, we need to clean up the unused indices, but we wanted to double sure that the old index is not used in any way.

We used the https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html, but it doesn't provide solid proof of whether the index was used recently or not.

Best solution would be some API from ES by which we can quickly determine and delete the old indices.


Solution

  • You can use the index stats API which allows you to see whether search or index requests have been handled by a given index since the last restart.

    GET index/_stats
    =>
      "_all": {
        "primaries": {
          "docs": {
            "count": 396885916,
            "deleted": 1712210
          },
          "store": {
            "size_in_bytes": 207383595268,
            "throttle_time_in_millis": 0
          },
          "indexing": {
            "index_total": 8,               <-- writes are happening if you see this increase
            "index_time_in_millis": 41,
            "index_current": 0,
            "index_failed": 0,
            "delete_total": 0,
            "delete_time_in_millis": 0,
            "delete_current": 0,
            "noop_update_total": 0,
            "is_throttled": false,
            "throttle_time_in_millis": 0
          },
          "get": {
            "total": 0,
            "time_in_millis": 0,
            "exists_total": 0,
            "exists_time_in_millis": 0,
            "missing_total": 0,
            "missing_time_in_millis": 0,
            "current": 0
          },
          "search": {
            "open_contexts": 7,
            "query_total": 30238,           <-- reads are happening if you see this increase
            "query_time_in_millis": 254000,
            "query_current": 0,
            "fetch_total": 816,
            "fetch_time_in_millis": 25997,
            "fetch_current": 0,
            "scroll_total": 7637,
            "scroll_time_in_millis": 686963111,
            "scroll_current": 7,
            "suggest_total": 0,
            "suggest_time_in_millis": 0,
            "suggest_current": 0
          },