nutchwarc

How to dump Nutch 2.3 data into WARC file?


I need to dump data from Nutch 2.3 into a WARC file. However, i couldn't find the necessary module. Nutch 1.x had this capability. I would like to know the proper way to do it.


Solution

  • As you said, at the moment the WARC exporter module is not yet ported to the 2.x branch of Nutch, nevertheless porting the https://github.com/apache/nutch/blob/master/src/java/org/apache/nutch/tools/warc/WARCExporter.java module shoudln't be that hard. As a general rule the 1.x branch of Nutch still is more used and better equiped than the 2.x branch (at least for now).