opensearchamazon-opensearch

Wait for shards to be refreshed


When indexing documents using the bulk endpoint, one can set the parameter refresh=wait_for in order to wait shards to be refreshed.

What I wanna do is disable index refreshing interval, send all bulk requests, call the refresh endpoint POST /{index}/_refresh at the end and then wait until all the refresh is done.

Is it possible to be done using OS's API?

I searched through the documentation but couldn't find a way of doing this.


Solution

  • You can disable the refresh operation by adjusting refresh_interval to -1. You can do for _all indices or only one of them.

    If you are trying to speed up the indexing speed you can also check the following documentation. https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html

    #disable the refresh
    PUT index/_settings
    {
      "refresh_interval": "-1"
    }
    
    #complete the bulk
    POST /index/_bulk
    
    #re-enable the refresh
    PUT index/_settings
    {
      "refresh_interval": null
    }
    
    #or just refresh manually
    POST index/_refresh
    

    index.refresh_interval How often to perform a refresh operation, which makes recent changes to the index visible to search. Defaults to 1s. Can be set to -1 to disable refresh. If this setting is not explicitly set, shards that haven’t seen search traffic for at least index.search.idle.after seconds will not receive background refreshes until they receive a search request. Searches that hit an idle shard where a refresh is pending will trigger a refresh as part of the search operation for that shard only. This behavior aims to automatically optimize bulk indexing in the default case when no searches are performed. In order to opt out of this behavior an explicit value of 1s should set as the refresh interval. https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-refresh-interval-setting