[SOLVED] Exporting index from Solr to a file, with fields illegal for FieldCache

Exporting index from Solr to a file, with fields illegal for FieldCache

I'm trying to export Solr index to a JSON file. However, among the 2 fields I care about, one of them (field A) is multivalued, another (field B) neither indexed nor has doc values (this field is probably missing schema). Both cause error that says can not use FieldCache on a field which is ...

The schemas of these fields is on remote server, and shouldn't be changed. Is it possible then to export the index with these fields anyway? Thank you!

p.s. If possible I would also like to fl on these 2 fields since they are all I need.

Solution

In that case you probably want to write an export script yourself, using a cursor mark to speed up retrieval (to use the /export feature, the fields has to have docValues enabled).

There's examples in a few languages for fetching all docs on the cursor mark page, and they can be applied almost directly (you'll have to add the JSON writing yourself) to your result set.

SolrQuery q = (new SolrQuery(some_query)).setRows(r).setSort(SortClause.asc("id"));
String cursorMark = CursorMarkParams.CURSOR_MARK_START;
boolean done = false;
while (! done) {
  q.set(CursorMarkParams.CURSOR_MARK_PARAM, cursorMark);
  QueryResponse rsp = solrServer.query(q);
  String nextCursorMark = rsp.getNextCursorMark();
  doCustomProcessingOfResults(rsp);
  if (cursorMark.equals(nextCursorMark)) {
    done = true;
  }
  cursorMark = nextCursorMark;
}

Make sure to use a rather large r value so the number of round trips is reduced.