Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Elastic search Multiple Aggregation

I have two elastic search queries

  1. For Get transaction information for a particular day
  2. For get transaction information for all days

How can i combine these two quires into a single query? Am struggling to write a signle query for these two similar needs. Please help me to solve this issue. Thank you

{
  "query": {
    "filtered": {
      "query": {
        "match": {
          "payment_type": "paypal"
        }
      },
      "filter": {
        "range": {
          "transaction_date": {
            "from": "2014-11-10",
            "to": "2014-11-10"
          }
        }
      }
    }
  },
  "aggs": {
    "daily_price_sum": {
      "sum": {
        "field": "price"
      }
    },
    "daily_post_sum": {
      "sum": {
        "field": "purchased_post_count"
      }
    }
  }
}


{
  "size": 0, 
  "query": {
    "match": {
      "payment_type": "paypal"
    }
  },
  "aggs": {
    "daily_price_sum": {
      "sum": {
        "field": "price"
      }
    },
    "daily_post_sum": {
      "sum": {
        "field": "purchased_post_count"
      }
    }
  }
}
like image 862
Dibish Avatar asked Nov 11 '14 07:11

Dibish


People also ask

Is Elasticsearch good for aggregation?

Elasticsearch Aggregations provide you with the ability to group and perform calculations and statistics (such as sums and averages) on your data by using a simple search query. An aggregation can be viewed as a working unit that builds analytical information across a set of documents.

What is sub aggregation in Elasticsearch?

Sub-aggregations allow you to continuously refine and separate groups of criteria of interest, then apply metrics at various levels in the aggregation hierarchy to generate your report.

How do you do aggregation in Kibana?

On the dashboard, click All types > Aggregation based, select Vertical bar, then select Kibana Sample Data Logs. Make sure the time filter is Last 7 days.

What is aggregation query?

An aggregate query is a method of deriving group and subgroup data by analysis of a set of individual data entries. The term is frequently used by database developers and database administrators.


1 Answers

If you are using ES of version less than 1.4.0, you can make use of Filter Aggregations. Query for the same is as below:

{
   "size": 0,
   "query": {
      "match": {
         "payment_type": "paypal"
      }
   },
   "aggs": {
      "daily_price_sum": {
         "sum": {
            "field": "price"
         }
      },
      "daily_post_sum": {
         "sum": {
            "field": "purchased_post_count"
         }
      },
      "one_day_aggs": {
         "filter": {
            "range": {
               "transaction_date": {
                  "from": "2014-11-10",
                  "to": "2014-11-10"
               }
            }
         },
         "aggs": {
            "daily_price_sum": {
               "sum": {
                  "field": "price"
               }
            },
            "daily_post_sum": {
               "sum": {
                  "field": "purchased_post_count"
               }
            }
         }
      }
   }
}

But if you are using ES 1.4.0, then you can use Filters Aggregation to make the query more compact. Query for the same is as below:

{
   "size": 0,
   "query": {
      "match": {
         "payment_type": "paypal"
      }
   },
   "aggs": {
      "transactions": {
         "filters": {
            "filters": {
               "one_day": {
                  "range": {
                     "transaction_date": {
                        "from": "2014-11-10",
                        "to": "2014-11-10"
                     }
                  }
               },
               "all_days": {
                  "match_all": {}
               }
            }
         },
         "aggs": {
            "daily_price_sum": {
               "sum": {
                  "field": "price"
               }
            },
            "daily_post_sum": {
               "avg": {
                  "field": "purchased_post_count"
               }
            }
         }
      }
   }
}

I also see that you are not interested in the query hits but only on the aggregation values. In that case, you can improve the performance of these aggregations by making use of Shard Query Cache which is present in ES 1.4.0. For making use of this, enable shard query cache as mentioned in the link and add the following parameter to the _search operation: search_type=count.

like image 191
bittusarkar Avatar answered Oct 22 '22 20:10

bittusarkar