Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to make solr synonyms work

Tags:

solr

synonym

I'm trying to set up some basic synonyms in Solr. The one I've been working on is:

us, usa, united states

My understanding is that adding that to the synonym file will allow users to search for US, and get back documents containing usa or united states. Ditto for if a user puts in usa or united states.

Unfortunately, with this in place, when I do a search, I get the results for items that contain all three of the words - it's doing an AND of the synonyms rather than an OR.

If I turn on debugging, this is indeed what I see (plus some stemming):

(+DisjunctionMaxQuery(((westCite:us westCite:usa westCite:unit) | (text:us text:usa text:unit) | (docketNumber:us docketNumber:usa docketNumber:unit) | ((status:us status:usa status:unit)^1.25) | (court:us court:usa court:unit) | (lexisCite:us lexisCite:usa lexisCite:unit) | ((caseNumber:us caseNumber:usa caseNumber:unit)^1.25) | ((caseName:us caseName:usa caseName:unit)^1.5))))/no_coord

Am I doing something wrong to cause this? My defaultOperator is set to AND, but I'd expect the synonym filter to understand that.

like image 416
mlissner Avatar asked Jan 14 '12 00:01

mlissner


People also ask

How does Solr work?

Solr works by gathering, storing and indexing documents from different sources and making them searchable in near real-time. It follows a 3-step process that involves indexing, querying, and finally, ranking the results – all in near real-time, even though it can work with huge volumes of data.


2 Answers

Try using the SynonymFilterFactory during indexing only, not during querying.

The documentation suggests this as well: http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory

like image 172
rfeak Avatar answered Sep 22 '22 19:09

rfeak


For better understanding of synonym search, Please follow step by step process of implementation below (I am using solr 6.5.* version):

Step 1:

Download country-synonyms.txt text file and place it in below path:

Path: \solr-6.5.1\server\solr\yourCore\conf

yourCore: Name of core should be changed accordingly

Step 2:

Add Field type in managed-schema file in same path mentioned above:

<fieldType name="country" class="solr.TextField" positionIncrementGap="100" sortMissingLast="true">
<analyzer>
  <tokenizer class="solr.KeywordTokenizerFactory"/>
  <filter class="solr.TrimFilterFactory"/>
  <filter class="solr.SynonymFilterFactory" expand="false" ignoreCase="true" synonyms="country-synonyms.txt" tokenizerFactory="solr.KeywordTokenizerFactory"/>
  <filter class="solr.LowerCaseFilterFactory"/>
</analyzer></fieldType>

Step 3: Add your field(Nationality) with type country in same file(managed-schema).

<field name="Nationality" type="country" indexed="true" stored="true"/>

Step 4: Restart solr.

solr restart -p <your solr port>

Step 5:

Now import your data with field containing Nationality.***

Step 6:

Now query with below cases and test:

Query:

  1. Nationality:US
  2. Nationality:USA
  3. Nationality:United States
  4. Nationality:United States of America

All above queries will give you same result.

Note:*** Import data only after performing above steps including solr restart. It may not work on existing data(For more details refer: AnalyzersTokenizersTokenFilters)

like image 30
shivadarshan Avatar answered Sep 22 '22 19:09

shivadarshan