Does Elasticsearch stream results?

Question

Does Elasticsearch stream the query results as they are "calculated" or does it calculate everything and then return the final response back to the client?

Answer 1

By default elasticsearch will only return a limited set of results for a query. (i.e. searching for * will only return the default count set regardless of the number of matches).

Generally to implement "streaming" , you make an initial search to get total count of matching documents and then ask for documents in ranges ( i.e. first 10, next 10, etc.. )

See

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-from-size.html

for how to request the number of documents returned.

Answer 2

Have you tried scroll query?

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html much easier to deal with than pagination.

Scrolling is not intended for real time user requests, but rather for processing large amounts of data, e.g. in order to reindex the contents of one index into a new index with a different configuration.