solrapache-zookeepersolrcloudsolrconfig

Solr storage handling


I have six node solr cluster and every node having 200GB of storage, we created one collection with two shards.

I like to know what will happen if my document reached 400GB (node1-200GB,node-2 200GB) ? is solr automatically use another free node from my cluster ?


Solution

    1. If my document reached 400GB (node1-200GB,node-2 200GB) ?

    Ans: I am not sure about what exacly error you may get, however in production you should try not to face this situation. To avoid/handle such scenarios we have monitoring/autoscaling triggers apis.

    1. Is solr automatically use another free node from my cluster ?

    Ans: No, Extra shards will not be added automatically. However whenever you observe that search is getting slow or if solr is crossing physical limitations of machines then you should go for splitShard .

    So ultimately you can handle this with autscaling triggers. That is you can set autscaling triggers to identify whether a shard is crossing specified limits about the number of document or size of the index etc. Once this limits reaches this trigger can call splitShard

    This link mentions

    This trigger can be used for monitoring the size of collection shards, measured either by the number of documents in a shard or the physical size of the shard’s index in bytes.

    When either of the upper thresholds is exceeded the trigger will generate an event with a (configurable) requested operation to perform on the offending shards - by default this is a SPLITSHARD operation.