Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Solr All checkers need to use the same Analyzer

Tags:

java

solr

I'm trying to run the spellcheck on some indexed data. I originally figure out how to do it but I messed with some configurations and now its saying that "All checkers need to use the same Analyzer." I've read that this is because the "field" values are named different things. I've made all the spellcheck components spell. What else do I doing wrong.

I get that error when I type is

http://localhost:8080/solr/collection1/select?q=name%3Adoc&wt=json&indent=true&spellcheck=true&spellcheck.collate=true&spellcheck.build=true

My solrconfig.xml


<requestHandler name="/select" class="solr.SearchHandler">

    <lst name="defaults">
        <!-- Optional, must match spell checker's name as defined above, defaults to "default" -->
        <str name="spellcheck.dictionary">default</str>
        <!-- Also generate Word Break Suggestions (Solr 4.0 see SOLR-2993) -->
        <str name="spellcheck.dictionary">wordbreak</str>
        <!-- omp = Only More Popular -->
        <str name="spellcheck.onlyMorePopular">false</str>
        <!-- exr = Extended Results -->
        <str name="spellcheck.extendedResults">false</str>
        <!--  The number of suggestions to return -->
        <str name="spellcheck.count">10</str>
    </lst>
    <arr name="last-components">
        <str>spellcheck</str>
    </arr>

</requestHandler>

<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
    <lst name="spellchecker">

        <str name="name">default</str>
        <!-- The classname is optional, defaults to IndexBasedSpellChecker -->
        <str name="classname">solr.IndexBasedSpellChecker</str>

        <str name="field">spell</str>
        <!-- Optional, by default use in-memory index (RAMDirectory) -->
        <str name="spellcheckIndexDir">./spellchecker</str>
        <!-- Set the accuracy (float) to be used for the suggestions. Default is 0.5 -->
        <str name="accuracy">0.7</str>
        <!-- Require terms to occur in 1/100th of 1% of documents in order to be included in the   dictionary -->
        <float name="thresholdTokenFrequency">.0001</float>
    </lst>
    <!-- a spellchecker that can break or combine words. (Solr 4.0 see SOLR-2993) -->
    <lst name="spellchecker">
        <str name="name">wordbreak</str>
        <str name="classname">solr.WordBreakSolrSpellChecker</str>
        <str name="field">spell</str>
        <str name="combineWords">true</str>
        <str name="breakWords">true</str>
        <int name="maxChanges">3</int>
    </lst>
    <!-- Example of using different distance measure -->
    <lst name="spellchecker">
        <str name="name">jarowinkler</str>
        <str name="field">spell</str>
        <!--  Use a different Distance Measure -->
        <str name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance</str>
        <str name="spellcheckIndexDir">./spellchecker</str>
    </lst>

    <!-- This field type's analyzer is used by the QueryConverter to tokenize the value for "q" parameter -->
    <str name="queryAnalyzerFieldType">textSpell</str>
</searchComponent>


<searchComponent class="solr.SpellCheckComponent" name="suggest">
    <lst name="spellchecker">
        <str name="name">spell</str>
        <str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
        <str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookup</str>

        <str name="field">suggest</str>
        <float name="threshold">0.005</float>
        <str name="buildOnCommit">true</str>

    </lst>

</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/suggest">
    <lst name="defaults">
        <str name="spellcheck">true</str>
        <str name="spellcheck.dictionary">suggest</str>
        <str name="spellcheck.onlyMorePopular">true</str>
        <str name="spellcheck.count">5</str>
        <str name="spellcheck.collate">true</str>
    </lst>

    <arr name="components">
        <str>suggest</str>
    </arr>
</requestHandler>

My schema.xml

<field name="id" type="string" indexed="true" stored="true" required="true"  multiValued="false" />
<field name="sku" type="text_en_splitting_tight" indexed="true" stored="true" omitNorms="true"/>
<field name="name" type="text_general" indexed="true" stored="true"/>
<field name="manu" type="text_general" indexed="true" stored="true" omitNorms="true"/>
<field name="cat" type="string" indexed="true" stored="true" multiValued="true"/>
<field name="features" type="text_general" indexed="true" stored="true" multiValued="true"/>
<field name="includes" type="text_general" indexed="true" stored="true" termVectors="true" termPositions="true" termOffsets="true" />

<field name="weight" type="float" indexed="true" stored="true"/>
<field name="price"  type="float" indexed="true" stored="true"/>
<field name="popularity" type="int" indexed="true" stored="true" />
<field name="inStock" type="boolean" indexed="true" stored="true" />

<field name="store" type="location" indexed="true" stored="true"/>

StackTrace

    java.lang.IllegalArgumentException: All checkers need to use the 
    same Analyzer.\n\tat org.apache.solr.spelling.ConjunctionSolrSpellChecker.addChecker(
    ConjunctionSolrSpellChecker.java:79)\n\tat org.apache.
    solr.handler.component.SpellCheckComponent.getSpellChecker(SpellCheckComponent.java:501)
    \n\tat org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.
    java:122)\n\tat org.apache.solr.handler.component.SearchHandler.handleRequestBody
    (SearchHandler.java:187)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest
    (RequestHandlerBase.java:135)\n\tat org.apache.solr.core.SolrCore.execute
    (SolrCore.java:1904)\n\tat org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659)
    \n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter
    (SolrDispatchFilter.java:362)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter
    (SolrDispatchFilter.java:158)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
    (ApplicationFilterChain.java:243)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter
    (ApplicationFilterChain.java:210)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke
    (StandardWrapperValve.java:222)\n\tat org.apache.catalina.core.StandardContextValve.invoke
    (StandardContextValve.java:123)\n\tat org.apache.catalina.core.StandardHostValve.invoke
    (StandardHostValve.java:171)\n\tat org.apache.catalina.valves.ErrorReportValve.invoke
    (ErrorReportValve.java:99)\n\tat org.apache.catalina.valves.AccessLogValve.invoke
    (AccessLogValve.java:953)\n\tat org.apache.catalina.core.StandardEngineValve.invoke
    (StandardEngineValve.java:118)\n\tat org.apache.catalina.connector.CoyoteAdapter.service
    (CoyoteAdapter.java:408)\n\tat org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
    \n\tat org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process
    (AbstractProtocol.java:589)\n\tat org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
    \n\tat java.util.concurrent.ThreadPoolExecutor$Worker.runTask
    (ThreadPoolExecutor.java:886)
    \n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\n\tat java.lang.Thread.run(Thread.java:662)\n",
        "code":500}}

Any help would be appreciated.

like image 576
Jaspreet Chauhan Avatar asked Sep 12 '13 22:09

Jaspreet Chauhan


2 Answers

The Error indicates the same field should be used by all the Spell Checkers defined.
In your Configuration you are using different fields (spell & suggest) for different checker.

<str name="field">spell</str> AND <str name="field">suggest</str>

like image 188
Jayendra Avatar answered Oct 21 '22 10:10

Jayendra


All fields, that are used in the spellchecker components (in the example spell and suggest), must use the same analysis chain. Analysis chain means they have to be configured with the same field type.

  <field name="spell" type="spellchecker" indexed="true" stored="true"/>
  <field name="suggest" type="spellchecker" indexed="true" stored="true"/>

  <fieldType name="spellchecker" class="solr.TextField">
    <analyzer>
      <tokenizer class="solr.StandardTokenizerFactory"/>
      <filter class="solr.LowerCaseFilterFactory"/>
    </analyzer>
  </fieldType>
like image 27
Matthias M Avatar answered Oct 21 '22 09:10

Matthias M