I have elastic cluster with hundreds of indices. Is there any way to list (search) indices using boolean query? e.g.
( index.alias:*read_index* AND doc.count:<1000 ) OR ( index.name* ) OR (index.size:<2gb) OR (index.replica:>2)
I need to filter out required indices from the list of hundreds of indices.
Kindly suggest.
You can query localhost:9200/_status and that will give you a list of indices and information about each.
You can use the search API to search and aggregate data stored in Elasticsearch data streams or indices. The API's query request body parameter accepts queries written in Query DSL. The following request searches my-index-000001 using a match query. This query matches documents with a user.id value of kimchy .
To search multiple data streams and indices, add them as comma-separated values in the search API's request path. The following request searches the my-index-000001 and my-index-000002 indices. You can also search multiple data streams and indices using an index pattern.
Boolean queries in Elasticsearch are a popular query type because of their versatility and ease of use. Boolean queries, or bool queries, find or match documents by using boolean clauses. For the vast majority of cases, the filtering clause will be used because it can be cached for faster search times.
Using plain elasticsearch bool queries :), just store the JSON format cat output into an index, then make the queries you need, automatize the collection with a cronjob to gather this every X time, my python script looks like this:
# install dependencies: pip install requests
import requests
import json
ES_URL = "http://localhost:9200"
res = requests.get("{}{}".format(ES_URL, "/_cat/indices"),
params={"format": "json", "bytes": "m"})
for index_info in res.json():
index_url = "{}/{}/{}/{}".format(
ES_URL, "cat_to_index", "doc", index_info["index"]
)
requests.post(
index_url,
data=json.dumps(index_info),
headers={'Content-type': 'application/json'}
)
# ready to query http://localhost:9200/cat_to_index/_search
# ready to keep up-to-date with a cronjob, as the index name is the ID new values will be overwritten.
hope it helps.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With