ruby-on-railssolrsunspot

EdgeNGramFilterFactory not working (not indexing?)


I am having trouble getting ngrams to work. Here's my schema.xml:

<fieldType name="text" class="solr.TextField" omitNorms="false">
  <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StandardFilterFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="25" />

  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StandardFilterFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true" />
  </analyzer>
</fieldType>

My database has a bunch of entries with

"Elizabeth"

and

"Elizabeths"

When I try to query on "Elizabeth" I get only "Elizabeth" and not "Elizabeths". The odd thing is, when I check out the solr admin, the Analysis page shows that the EdgenGramFilterFactory is indeed available, and results in "Elizabeths" being expanded into

e el eli eliz eliza elizab elizabe elizabet elizabeth

It seems like the indexer isn't picking up on this. I have the same problem when I move the synonyms filter from the query block to the index block. That is to say, when I have the synonyms filter in the query block, it works, but when I put it in the index block, it has no effect.

I have restarted Sunspot and reindexed multiple times. No dice. Any ideas? How can I directly check the indexed words list?


Solution

  • I think I found the problem and it looks like a noob error.

    In my model, is was using the following construct as per one of the tutorials:

    class Institution < ActiveRecord::Base
     .
     .
     .
    end
    
    Sunspot.setup(Institution) do
      text :name
    end
    

    This did not seem to throw any errors when I started, stopped, or reindexed. It struck me as strange that I was able to reindex immediately after stopping Solr.

    I switched to

    class Institution < ActiveRecord::Base
      .
      .
      .
      searchable do
        text :name
      end
    endH
    

    When I did this, I found that I could not reindex after stopping Solr. However, when I started Solr and reindexed, the index appeared to be truly refreshed and my queries finally behaved as expected.