Alternatives for real time score by popularity with elasticsearch

Question

I would like boost a document's score by popularity. I'd like it to be as real-time as possible.

In order to meet the real time requirement, it seems I have to re-index each document each time it's popularity changes (per view). This seems highly inefficient.

An alternative is to run a batch process that periodically re-indexes documents that have been recently viewed, but this becomes less real-time, and still requires re-indexing entire documents when only one field (the popularity) has changed.

A third approach (which we have implemented) is to use a plugin to grab a document's popularity from an external source and use a script to include it in scoring. This works as well, but slows down search for large document spaces. Using rescore helps, but it only allows us to sort a subset of the documents returned.

Is there a better option (a way to add popularity to the index without reindexing the entire document or a better way to integrate external data with elastic search)?

Answer 1

You can try the following to have realtime popularity field.

1. Include a popularity field as part of your index.

2. Increment popularity every time a document is retrieved. You can do this using partial update scripts.

3. Use function score query to boost the document.

Java API:

new FunctionScoreQueryBuilder(matchQuery("canonical_name",
                                phrase).analyzer("standard")
                                .minimumShouldMatch("100%")).add(
                                fieldValueFactorFunction("popularityScore")
                                        .modifier(Modifier.LOG1P).factor(2f))
                                .boostMode("sum"))

http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/boosting-by-popularity.html