Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

hits>total - Limited to 10000 records - Increase the limit

enter image description here

My API request looks like this... GET /my_index/_search?scroll=1m

Along with getting the search results, I'm also looking to get the count of the total results at one go. However, while targeting hits>total>value, the number of records never show a count more than 10000. Due to this, I have to fire count API separately. Is there any way where I can get more than 10000 records in my same _search query?

like image 811
Abhilash Avatar asked Feb 29 '20 17:02

Abhilash


People also ask

How do I get more than 10000 hits in Elasticsearch?

By default, you cannot use from and size to page through more than 10,000 hits. This limit is a safeguard set by the index. max_result_window index setting. If you need to page through more than 10,000 hits, use the search_after parameter instead.

What is hits in elastic search?

A search consists of one or more queries that are combined and sent to Elasticsearch. Documents that match a search's queries are returned in the hits, or search results, of the response. A search may also contain additional information used to better process its queries.

How do I set Elasticsearch limits?

So basically instead of limiting from or size (or a combination of those), you set max_result_window to 1000 and ES will only return a maximum of 1000 hits per request. If you are using an index definition in a separate JSON file to create your index, you can set this value there under yourindexname. settings.


2 Answers

Simply add "track_total_hits": true to your request.

(see Elasticsearch Reference: Track total hits)

like image 180
Daniel Schneiter Avatar answered Oct 10 '22 23:10

Daniel Schneiter


Try to set track_total_hits search option to true.

Generally the total hit count can’t be computed accurately without visiting all matches, which is costly for queries that match lots of documents. The track_total_hits parameter allows you to control how the total number of hits should be tracked. Given that it is often enough to have a lower bound of the number of hits, such as "there are at least 10000 hits", the default is set to 10,000. This means that requests will count the total hit accurately up to 10,000 hits. It’s is a good trade off to speed up searches if you don’t need the accurate number of hits after a certain threshold.

When set to true the search response will always track the number of hits that match the query accurately

There is a great article from the official documentation describing what it is.

Or, if you only need total count, just use Count API:

like image 26
Oleksii Miroshnyk Avatar answered Oct 10 '22 23:10

Oleksii Miroshnyk