I have a classifieds website. Users may put ads, edit ads, view ads etc.
Whenever a user puts an ad, I am adding a document to Solr. I don't know, however, when to commit it. Commit slows things down from what I have read.
How should I do it? Autocommit every 12 hours or so?
Also, how should I do it with optimize?
Actually, committing often and optimizing makes things really slow. It's too heavy.
After a day of searching and reading stuff, I found out this:
1- Optimize causes the index to double in size while beeing optimized, and makes things really slow.
2- Committing after each add is a bad idea, it can really slow down your server and/or stretch the time of our loading process for hours. It's better commit when a reasonable amount of documents are sent to Solr (100, 500, 5000?), this should be tuned according your needs. You can even commit only when the last document of your update is sent to Solr. On the other hand you can run the optimize task when the load of your server is really low, once a day could be enough.
3- Commit should be set to "autoCommit" in the solrconfig.xml file, and also this should be tuned according to your needs.