Python elasticsearch.helpers.scan example

Question

Can someone provide scan API example of python elasticsearch helpers client?

res = elasticsearch.helpers.scan(....)

How can i get all results from elasticsearch with res object?

Answer 1

The documentation includes an example, although if I'm reading it right, helpers.scan by default sets search_type=scan, which was removed in ES 5.1. This causes the example code to fail with ES returning No search type for [scan]. We can amend this with preserve_order=True (I am however not sure about the performance implications here):

import elasticsearch
import elasticsearch.helpers
es = elasticsearch.Elasticsearch()
results = elasticsearch.helpers.scan(es,
    index="test_index",
    doc_type="my_document",
    preserve_order=True,
    query={"query": {"match_all": {}}},
)

for item in results:
    print(item['_id'], item['_source']['name'])

This helper returns an object which you can iterate to obtain the actual results from the query.

item is of form

{'_index': <str>, '_type': <str>, '_id': <str>, '_score': <float or None>, '_source': {'key': val}, 'sort': [<int>]}