Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to query and list all types within an elasticsearch index?

Tags:

Problem: What is the most correct way to simply query for and list all types within a specific index (and all indices) in elasticsearch?

I've been reading through the reference and API but can't seem to find anything obvious.

I can list indices with the command:

$ curl 'localhost:9200/_cat/indices?v'

I can get stats (which don't seem to include types) with the command:

$ curl localhost:9200/_stats

I'd expect that there'd be a straightforward command as simple as:

$ curl localhost:9200/_types

or

$ curl localhost:9200/index_name/_types

Thanks for any help you can offer.

like image 831
Information Technology Avatar asked Dec 08 '15 01:12

Information Technology


People also ask

How do I list all indices in Elasticsearch?

There are multiple ways to list all of the indexes contained in an Elasticsearch cluster. One common way of doing this is to use cURL and Kibana to issue HTTP requests to communicate with the Elasticsearch cluster to have it return all of the index names and their respective UUIDs.

How do I query multiple indices in Elasticsearch?

To search multiple data streams and indices, add them as comma-separated values in the search API's request path. The following request searches the my-index-000001 and my-index-000002 indices. You can also search multiple data streams and indices using an index pattern.

How do I get Elasticsearch index data?

You can use the search API to search and aggregate data stored in Elasticsearch data streams or indices. The API's query request body parameter accepts queries written in Query DSL. The following request searches my-index-000001 using a match query. This query matches documents with a user.id value of kimchy .

Can index have multiple types Elasticsearch?

From Elasticsearch version 6.0 by default index doesn't allow multiple types per index. The better option is to always have one document type per index. The single responsibility of the index is to maintain the single document type/mapping type per index.


2 Answers

What you call "type" is actually a "mapping type" and the way to get them is simply by using:

curl -XGET localhost:9200/_all/_mapping

Now since you only want the names of the mapping types, you don't need to install anything, as you can use simply use Python to only get you what you want out of that previous response:

curl -XGET localhost:9205/_all/_mapping | python -c 'import json,sys; indices=json.load(sys.stdin); indices = [type for index in indices for type in indices.get(index).get("mappings")]; print list(indices);'

The Python script does something very simple, i.e. it iterates over all the indices and mapping types and only retrieves the latter's names:

import json,sys; 
resp = json.load(sys.stdin); 
indices = [type for index in resp for type in indices.get(index).get("mappings")]; 
print list(indices);'

UPDATE

Since you're using Ruby, the same trick is available by using Ruby code:

curl -XGET localhost:9205/_all/_mapping | ruby -e "require 'rubygems'; require 'json'; resp = JSON.parse(STDIN.read); resp.each { |index, indexSpec | indexSpec['mappings'].each {|type, fields| puts type} }"

The Ruby script looks like this:

require 'rubygems';
require 'json';
resp = JSON.parse(STDIN.read);
resp.each { |index, indexSpec | 
    indexSpec['mappings'].each { |type, fields| 
        puts type
    }
}
like image 159
Val Avatar answered Sep 22 '22 04:09

Val


You can just print the index and use the _mapping API so you will see only the section of "mappings" in the index.

For example: curl -GET http://localhost:9200/YourIndexName/_mapping?pretty

You will get something like that:

{
"YourIndexName" : {
    "mappings" : {
      "mapping_type_name_1" : {
        "properties" : {
          "dateTime" : {
            "type" : "date"
          },
          "diskMaxUsedPct" : {
            "type" : "integer"
          },
          "hostName" : {
            "type" : "keyword"
          },
          "load" : {
            "type" : "float"
          },
          "memUsedPct" : {
            "type" : "float"
          },
          "netKb" : {
            "type" : "long"
          }
        }
      },
      "mapping_type_name_2" : {
        "properties" : {
          "dateTime" : {
            "type" : "date"
          },
          "diskMaxUsedPct" : {
            "type" : "integer"
          },
          "hostName" : {
            "type" : "keyword"
          },
          "load" : {
            "type" : "float"
          },
          "memUsedPct" : {
            "type" : "float"
          }
        }
      }
    }
  }
}

mapping_type_name_1 and mapping_type_name_2 are the types in this index, and you also can see the structure of these types.

Good explanation about mapping_types is here: https://logz.io/blog/elasticsearch-mapping/

like image 23
RafaelJan Avatar answered Sep 24 '22 04:09

RafaelJan