Getting stable SOLR scores

Question

I run a query against a SOLR core and restrict the result using a filter like fq: {!frange l=0.7 }query($q). I'm aware that SOLR scores do not have an absolute meaning, but the 0.7 (just an example) is calculated based on user input and some heuristics, which works quite well.

The problem is the following: I update quite a few documents in my core. The updated fields are only meta data fields, which are unrelated to the above search. But because an update is internally a delete + insert, IDF and doc counts change. And so do the calculated scores. Suddenly my query returns different results.

As Yonik explained to me here, this behaviour is by design. So my question is: What is the most simple and minimal way to keep the scores and the output of my query stable?

Running optimize after each commit should solve the problem, but I wonder if there is something simpler and less expensive.

Answer 1

You really need to run optimize. When you optimize the index solr clean all documents not pointed yet and make the query stable. This occurs because build this meta data information is expensive to be done all the time a document is updated. Because of this solr just do that on optimize. There is a good way to see if your index is more or less stable... When you access Solr API you could see Num Docs and Max Doc information. If Max Doc is greater than Num Docs it seams that you have some old products affecting your relevancy calculation. Optimizing the index these two numbers is made equal again. If these numbers are equal you can trust IDF is been calculated correctly.