solrsolr6

Searching other fields with Solr hon-lucene-synonyms plugin?

I'm trying to figure out how to use this plugin:

https://github.com/healthonnet/hon-lucene-synonyms

If I run:

http://solr.example.com/solr/graphics/select/?q=royalty+free&debugQuery=on&defType=synonym_edismax&synonyms=true

It works as I want, and I get the debugging telling me that its doing what I want:

<arr name="expandedSynonyms">
<str>art</str>
<str>cartoon</str>
<str>clip</str>
<str>clipart</str>
<str>graphics</str>
<str>image</str>
<str>images</str>
<str>multimedia</str>
<str>picture</str>
<str>pictures</str>
<str>royalty free</str>
</arr>

+(((_text_:royalty) (_text_:free))^1.0 ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:cartoons _text_:clip _text_:clipart _text_:comic _text_:draw _text_:drawing _text_:drawings _text_:funny _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty _text_:sketch) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0) ((+((Synonym(_text_:art _text_:cartoon _text_:clip _text_:clipart _text_:graphics _text_:image _text_:images _text_:multimedia _text_:picture _text_:pictures _text_:royalty) _text_:free)))^1.0))

The problem comes when I also want to narrow the results down more using other filters:

http://solr.example.com/solr/graphics/select/?q=({!lucene%20sow=false%20df=title}%20royalty+free)%20AND%20(has_fla:1)&&debugQuery=on&defType=synonym_edismax&synonyms=true

I just get this debugging:

<lst name="reasonForNotExpandingSynonyms">
<str name="name">HasComplexQueryOperators</str>
<str name="explanation">
synonyms.ignoreQueryOperators is set to false, and this query contains complex query operators (e.g. AND, OR, *, -, etc.). Complex queries aren't supported.
</str>
</lst>

Surely there must be a way to keep the synonym stuff working, whilst also searching on other fields as well? I'm using Solr 6.6.0

The QueryParser is solrconfig.xml looks like:

 <queryParser name="synonym_edismax" class="com.github.healthonnet.search.SynonymExpandingExtendedDismaxQParserPlugin">
    <!-- You can define more than one synonym analyzer in the following list.
         For example, you might have one set of synonyms for English, one for French,
         one for Spanish, etc.
      -->
    <lst name="synonymAnalyzers">
      <!-- Name your analyzer something useful, e.g. "analyzer_en", "analyzer_fr", "analyzer_es", etc.
           If you only have one, the name doesn't matter (hence "myCoolAnalyzer").
        -->
      <lst name="myCoolAnalyzer">
        <!-- We recommend a PatternTokenizerFactory that tokenizes based on whitespace and quotes.
             This seems to work best with most people's synonym files.
             For details, read the discussion here: http://github.com/healthonnet/hon-lucene-synonyms/issues/26
          -->
        <lst name="tokenizer">
          <str name="class">solr.PatternTokenizerFactory</str>
          <str name="pattern"><![CDATA[(?:\s|\")+]]></str>
        </lst>
        <!-- The ShingleFilterFactory outputs synonyms of multiple token lengths (e.g. unigrams, bigrams, trigrams, etc.).
             The default here is to assume you don't have any synonyms longer than 4 tokens.
             You can tweak this depending on what your synonyms look like. E.g. if you only have unigrams, you can remove
             it entirely, and if your synonyms are up to 7 tokens in length, you should set the maxShingleSize to 7.
          -->
        <lst name="filter">
          <str name="class">solr.ShingleFilterFactory</str>
          <str name="outputUnigramsIfNoShingles">true</str>
          <str name="outputUnigrams">true</str>
          <str name="minShingleSize">2</str>
          <str name="maxShingleSize">4</str>
        </lst>
        <!-- This is where you set your synonym file.  For the unit tests and "Getting Started" examples, we use example_synonym_file.txt.
             This plugin will work best if you keep expand set to true and have all your synonyms comma-separated (rather than =>-separated).
          -->
        <lst name="filter">
          <str name="class">solr.SynonymFilterFactory</str>
          <str name="tokenizerFactory">solr.KeywordTokenizerFactory</str>
          <str name="synonyms">synonyms.txt</str>
          <str name="expand">true</str>
          <str name="ignoreCase">true</str>
        </lst>
      </lst>
    </lst>
  </queryParser>

For what it's worth - the reason we are using this plugin, is because we want to use multiple word synonyms - such as:

royalty free, cartoon, images, photos

With a standard setup "royalty" and "free" and treated as 2 seperate words in the synonyms, which we dont want.

Thanks

Solution

  • some comments: