stormcrawler

Replacement of ESSeedInjector in storm-crawler 2.2


I'm updating our crawler from storm-crawler 1.14 to 2.2. What is the replacement for the old ESSeedInjector?


Solution

  • The class-based topologies have been replaced by Flux files, which are far more flexible to use. The injection is now done as part of the crawl as you can see in es-crawler.flux. It would be easy to extract the injection part and put that in a separate script if you want to keep things separate. Alternatively, you could copy the code back from 1.14, put it in your project and fix whatever needs fixing for it to work with Storm 2.x.