Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Elastic Search size to unlimited

Am new to elastic search. Am facing a problem to write a search query returning all matched records in my collection. Following is my query to search record

{
   "size":"total no of record" // Here i need to get total no of records in collection
   "query": {
      "match": {
         "first_name": "vineeth"
      }
   }
}

By running this query i am only getting maximum 10 records, am sure there is more than 10 matching records in my collection. I searched a lot and finally got size parameter in query. But in my case i dont know the total count of records. I think giving an unlimited number to size variable is not a good practice, so how to manage this situation please help me to solve this issue, Thanks

like image 212
Dibish Avatar asked Mar 14 '14 12:03

Dibish


3 Answers

It's not very common to display all results, but rather use fromand size to specify a range of results to fetch. So your query (for fetching the first 10 results) should look something like this:

{
    "from": 0,
    "size": 10,
    "query": {
        "match": {
            "first_name": "vineeth"
        }
    }
}

This should work better than setting size to a ridiculously large value. To check how many documents matched your query you can get the hits.total (total number of hits) from the response.

like image 138
dbrumann Avatar answered Oct 13 '22 06:10

dbrumann


To fetch all the records you can also use scroll concept.. It's like cursor in db's..
If you use scroll, you can get the docs batch by batch.. It will reduce high cpu usage and also memory usage..

For more info refer

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html

like image 39
BlackPOP Avatar answered Oct 13 '22 05:10

BlackPOP


To get all records, per de doc, you should use scroll. Here is the doc: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html

But the idea is to specify your search and indicate that you want to scroll it:

curl -XGET 'localhost:9200/twitter/tweet/_search?scroll=1m' -d '
{
   "query": {
    "match" : {
        "title" : "elasticsearch"
     }
 }
}'

in the scroll param you specify how long you want the search results available.

Then you can retrieve them with the returned scroll_id and the scroll api.

like image 3
Santiago Trias Avatar answered Oct 13 '22 05:10

Santiago Trias