Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Aggregation + sorting +pagination in elastic search

I need to do an aggregation + sorting + pagination in one of the indexes.

I learned about internal functionality of Elastic search,

I have 5 total shards, it will sort the individual shards and fetch the result, by default each shard will return in 10 records. Then the 50 records are sorted again and it will fetch the top 10 record since by default size is 10.

ouput:

The aggregated results are returned in separate field named as "aggregations".In order to do pagination in this aggregated data,size and from are not working.

So tired of doing termBuilder.size(500), now the logic was differs as per this link (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html)

It leads to inaccuracy of data.

Can any one suggest me how to deal with aggregation + pagination.

like image 297
kumar Avatar asked Jan 05 '15 09:01

kumar


People also ask

What is aggregation in Elasticsearch?

In Elasticsearch, an aggregation is a collection or the gathering of related things together. The aggregation framework collects data based on the documents that match a search request which helps in building summaries of the data.

How does pagination work in Elasticsearch?

The simplest method of pagination uses the from and size parameters available in Elasticsearch's search API. By default, from is 0 and size is 10, meaning if you don't specify otherwise, Elasticsearch will return only the first ten results from your index.

When exploring records sets what is the best approach for pagination?

So if you're looking for a quick way to implement pagination then offset is the way to go. Especially if that data is not real-time. No need to complicate things where it is not needed. Having implemented cursors myself, I would say that it wasn't too hard to get it working for 90% of cases.


1 Answers

In elasticsearch, it's not possible to paginate an aggregation. The query will not give accurate results if size is specified. So, the only way to do sorting and pagination is to give size 0 and return all the documents and then, get the required results by accumulating all the results in a list for further operation.

like image 81
Nishant Avatar answered Sep 30 '22 15:09

Nishant