How to get all indices with Elastic's High Level Rest Client?

Question

I want a nice, quick and easy way to get all of the indices in elasticsearch using the their Java REST client. I am currently able to do this by grabbing their lower level client, like this:

public void fetchIndices() throws IOException {
    List<String> indices = null;

    RestClient restClient = client.getLowLevelClient();
    Response response = null;
    try {
        response = restClient.performRequest("GET", "/_cat/indices?v");
    } catch (IOException e) {
        LOGGER.log(Level.WARNING, e.toString(), e);
    }

    InputStream inputStream = null;
    if (response != null) {
        try {
            inputStream = response.getEntity().getContent();
        } catch (IOException e) {
            LOGGER.log(Level.WARNING, e.toString(), e);
        }
    }

    if (inputStream != null) {
        InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
        BufferedReader bufferedReader = new BufferedReader(inputStreamReader);

        indices = new ArrayList<>();
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            // Get tokens with no whitespace
            String[] tokens = line.split("\\s+");
            for (String token : tokens) {
                // TODO - make the startsWith() token configurable
                if (token.startsWith(SOME_TOKEN)) {
                    LOGGER.log(Level.INFO, "Found elasticsearch index " + token);
                    indices.add(token);
                    break;
                }
            }
        }
    }

    // Only update if we got data back from our REST call
    if (indices != null) {
        this.indices = indices;
    }
}

Essentially I just call the /_cat/indices?v endpoint as recommended in their docs. This works fine, but I was wondering if there was a nicer way to do this using the Java API. I can't seem to find a way in their current API, but wondering if anyone knows something I don't. Having to work with InputStreams and the various Readers isn't necessarily terrible, but just want to clean up the hacky string parsing.

Answer 1

As of Elasticsearch 7.5.0 you can use the following to retrieve all indices:

    GetIndexRequest request = new GetIndexRequest("*");
    GetIndexResponse response = client.indices().get(request, RequestOptions.DEFAULT);
    String[] indices = response.getIndices();

Answer 2

Try using: /_cat/indices?h=i

InputStream inputStream = restHighLevelClient.getLowLevelClient()
.performRequest("GET", "/_cat/indices?h=i")
.getHttpResponse()
.getEntity()
.getContent();

List<String> indexes = new BufferedReader(new InputStreamReader(inputStream))
    .lines()
    .collect(Collectors.toList());

Also, if you want to search using a regex: /_cat/indices?h=i&index=test*

Answer 3

At the moment, the high-level REST client doesn't support this. You can keep calling the _cat/indices API using the low-level client but try adding &format=json in the query string parameters. That way you'll get the same information but formatted as JSON which is much easier to parse (e.g. with the Jackson library):

List<String> indices = null;

RestClient restClient = client.getLowLevelClient();
Response response = null;
try {
    response = restClient.performRequest("GET", "/_cat/indices?v&format=json");
} catch (IOException e) {
    LOGGER.log(Level.WARNING, e.toString(), e);
}

// parse the JSON response
List<Map<String, String>> list = null;
if (response != null) {
    String rawBody = EntityUtils.toString(response.getEntity());
    TypeReference<List<HashMap<String, String>>> typeRef = new TypeReference<List<HashMap<String, String>>>() {};
    list = mapper.readValue(rawBody, typeRef);
}

// get the index names
if (list != null) {
    indices = list.stream()
        .map(x -> x.get("index"))
        .collect(Collectors.toList());
}

// Only update if we got data back from our REST call
if (indices != null) {
    this.indices = indices;
}

Note: Here is the roadmap for the High-Level REST client: https://github.com/elastic/elasticsearch/issues/27205

Answer 4

on es 6.8, use * or _all in GetIndexRequest when there is no index throws NoSuchIndexException.

I found this is the safer, no throw way:

    GetMappingsResponse response = esClient.indices().getMapping(new GetMappingsRequest().indices("*"),
            RequestOptions.DEFAULT);
    return new ArrayList<>(response.mappings().keySet());