solrdataimporthandler

How can I be notified in real-time when Solr is ready to accept another import request?


I'm writing a simple queue that sends update requests to a Solr core's DataImportHandler. This handler updates the Solr core by running a query against the database. When one update is complete, I'd like to immediately send the next update request. However, I'm having some trouble detecting when Solr is ready to accept another update request. Here's what I've tried:

Is there another way to detect when Solr is ready to accept another update request?


Solution

  • What I have done in similar scenarios:

    1. add more DIH handlers, you can have as many as you want, each one pointing to it's own xml config file if necessary. Add 10 of these for example
    2. each time you have some piece of data that needs a DIH handler, iterate over all of them ill you find one free (add some sleep() for sanity if all are busy). This has worked me well for large volume indexing.

    Of course this needs you indexing operations to be parallelizable, if they aren't this setup won't work.

    By the way, postOptimize would be called when you call optimize, not commit.