elasticsearchmetricbeat

Best practice to delete old MetricBeats data in ElasticSearch

How do we best delete old data collected via MetricBeats in ElasticSearch periodically? Is there some feature to automatically purge old data or to create new MetricBeats indices after a certain configurable period of time so those old indices can be deleted? I can see following index being created by metricbeat, it has a date in its name. So, is there a way that it can create new indices after a certain period? 

health status index                              pri rep docs.count docs.deleted store.size pri.store.size
yellow open   metricbeat-7.6.2-2020.05.06-000001 1   1    4120157            0    836.5mb        836.5mb
Solution

  • Using Index LifeCycle Policies looks like best approach to purge old data from Metricbeat indices. You can create or find one in Management Panel in Kibana as shown below:

    enter image description here

    I found a default policy was already created for MetricBeat. Edit the policy as per your liking. One can choose max index size for rollover, maximum Age and so on.

    enter image description here There is a delete phase at the bottom. Enable the same for automatic deletion of old data.

    enter image description here