Custom Per-Field Similarity in Solr4.1

Custom Per-Field Similarity in Solr4.1

Lets say a search term appears 20 times in a field “long description” of the solr document, it appears 5 times in field “short description”, due to the nature of data contained in field “short description”, we want to add a multiplier to the term frequency (tf) that Lucene Similarity calculates.

E.g. the tf score returned for “short description” should be multiplied by 10. For “long description” we dont want to give any additional score if there are more than 10 occurrences of the search term; the logic being, more occurrences beyond a threshold does not necessarily make it more relevant.

Solr provides a way to override the default Lucene Similarity class by specifying under schema.xml as mentioned below:

<similarity>com.sdudhara.MyCustomSImilarity</similarity>

In the MyCustomSImilarity class that extends from DefaultSimilarity (or any other Lucene Similarity class) , you can override the methods in the DefaultSimilarity class e.g. tf(). This will however impact all the fields and use same logic to calculate tf() score for all the fields.

To do this at the field level, in the schema.xml, go to the fieldType where you have defined the fieldType for the given field. Within the fieldType, you can add one more line with <similiarity>com.sdudhara.MyCustomSimilarity</similarity>

http://wiki.apache.org/solr/SchemaXml#Similarity

 <fieldType name="text_dfr" class="solr.TextField">
    <analyzer class="org.apache.lucene.analysis.standard.StandardAnalyzer"/>
    <similarity class="solr.DFRSimilarityFactory">
      <str name="basicModel">I(F)</str>
      <str name="afterEffect">B</str>
      <str name="normalization">H2</str>
    </similarity>
 </fieldType>
 <fieldType name="text_ib" class="solr.TextField">
    <analyzer class="org.apache.lucene.analysis.standard.StandardAnalyzer"/>
    <similarity class="solr.IBSimilarityFactory">
      <str name="distribution">SPL</str>
      <str name="lambda">DF</str>
      <str name="normalization">H2</str>
    </similarity>
 </fieldType>

• You will also need to delete existing data, reindex the data, since Similarity class is also used during index times. So, unless you reindex the data, you wont be able to get the custom similarities take effect

A related StackOverflow link that I had posted to resolve the issue:

http://stackoverflow.com/questions/15751766/solr-4-1-dismax-pf-not-returning-expected-results/15868556#15868556

Read full article from Custom Per-Field Similarity in Solr4.1