Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get a count for each type of an index in elasticsearch?

I have an index and want to get a count for the entries in every type of one particular index in elasticsearch, but might not know the types ahead of time.

So, for example, the index is

/events

and the types could be

/events/type1
/events/type2
...
/events/typeN

And I'd like to query the index and say "Give me the count of each of the types under index events", so maybe a result set like

/events/type1 : 40
/events/type2: 20
/events/typeN: 10

where /events/_count would give me

/events: 70

Edit:

imotov's answer is great. I'm having trouble figuring how to get it working in JavaScript/Ajax easily though. I have something like this right now:

$.ajax({
type: 'GET',
url: 'http://localhost:9200/events/_search?search_type=count',
data: '{ "facets" : { "count_by_type" : { "terms" : { "field": "_type" }}}}',
success: function(text) {
    console.log(text);
}
)}'

But am only getting the total count of elements in the ES, the facets portion of the answer seems to be missing.

like image 721
cdietschrun Avatar asked Jul 09 '13 20:07

cdietschrun


People also ask

How do I get a list of all indices in Elasticsearch?

There are multiple ways to list all of the indexes contained in an Elasticsearch cluster. One common way of doing this is to use cURL and Kibana to issue HTTP requests to communicate with the Elasticsearch cluster to have it return all of the index names and their respective UUIDs.

Can index have multiple types Elasticsearch?

You can't. One type per index only.

What is the difference between index and type in Elasticsearch?

In RDBMS terms, index is a database and type can be a table which contains many rows( document in elasticsearch).


4 Answers

You can use terms aggregations on the _type field to get this information:

curl "localhost:9200/test-idx/_search?search_type=count" -d '{
    "aggs": {
        "count_by_type": {
            "terms": {
                "field": "_type"
            }
        }
    }
}'
like image 187
imotov Avatar answered Oct 17 '22 10:10

imotov


For Elasticsearch v5.0, search_type=count is removed. The same query from the above answers can be written as follows:

GET  indexname/_search
{
    "aggs": {
        "count_by_type": {
            "terms": {
                "field": "_type"
            }
        }
    },
    "size": 0
}

Ref: https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking_50_search_changes.html#_literal_search_type_count_literal_removed

like image 26
Akshay Bijawe Avatar answered Oct 17 '22 12:10

Akshay Bijawe


The "facets" are deprecated in ES v. 1.5+ However you can use "aggregations", the use and results are quite similar:

curl "localhost:9200/events/_search?search_type=count" -d '{
    "aggregations": {
        "count_by_type": {
            "terms": {
                "field": "_type"
            }
        }
    },
    "size": 0
}'

You'll get something like:

{
   "took": 21,
   "timed_out": false,
   "_shards": {
      "total": 10,
      "successful": 10,
      "failed": 0
   },
   "hits": {
      "total": 150,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "count_by_type": {
         "doc_count_error_upper_bound": 0,
         "sum_other_doc_count": 0,
         "buckets": [
            {
               "key": "type1",
               "doc_count": 141
            },
            {
               "key": "type2",
               "doc_count": 6
            },
            {
               "key": "other_type",
               "doc_count": 3
            }
         ]
      }
   }
}
like image 42
Roberto Avatar answered Oct 17 '22 11:10

Roberto


Answers by @Askshay and @Roberto highlight another important aspect. Setting size to 0 is really important, especially in low-bandwidth use-cases (say in mobile networks). It reduces the data payload size and that makes a huge difference when document size is large. Note "size": 0

GET  index/_search
{
    "aggs": {
        "countByType": {
            "terms": {
                "field": "_type"
            }
        }
    },
    "size": 0
}
like image 44
Srihari Sridharan Avatar answered Oct 17 '22 10:10

Srihari Sridharan