Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Elastic Search (COUNT*) with group by and where condition

Dear Elastic Serach users,

I am newbie in ElasticSearch.

I am confused for how to convert the following sql command into elasticSearch DSL query ? Can anyone help to assist me.

SELECT ip, count(*) as c  FROM elastic WHERE  date 
BETWEEN '2016-08-20  00:00:00' and '2016-08-22 13:41:09' 
AND service='http' AND destination='10.17.102.1' GROUP BY ip ORDER BY c DESC;

THank YOu

like image 272
user647527 Avatar asked Aug 24 '16 03:08

user647527


People also ask

How do I count unique values in Elasticsearch?

There's no support for distinct counting in ElasticSearch, although non-deterministic counting exists. Use "terms" aggregation and count buckets in result. See Count distinct on elastic search question.

How check count in Kibana?

Create "topN" query on "clientip" and then histogram with count on "clientip" and set "topN" query as source. Then you will see count of different ips per time.

How do Elasticsearch aggregations work?

Elasticsearch Aggregations provide you with the ability to group and perform calculations and statistics (such as sums and averages) on your data by using a simple search query. An aggregation can be viewed as a working unit that builds analytical information across a set of documents.


1 Answers

The following query will achieve exactly what you want, i.e. it will select the documents within the desired date range and with the required service and destination and then run a terms aggregation (=group by) on their ip field and order the latter in decreasing count order.

{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "date": {
              "gt": "2016-08-22T00:00:00.000Z",
              "lt": "2016-08-22T13:41:09.000Z"
            }
          }
        },
        {
          "term": {
            "service": "http"
          }
        },
        {
          "term": {
            "destination": "10.17.102.1"
          }
        }
      ]
    }
  },
  "aggs": {
    "group_by_ip": {
      "terms": {
        "field": "ip"
      }
    }
  }
}
like image 163
Val Avatar answered Oct 25 '22 15:10

Val