Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to filter an elasticsearch global aggregation?

What I want to achieve: I want my "age" aggregation to not be filtered by the query filter and I want to be able to apply filters to it.

So if I start with this query:

{
    "query":{
        "filtered":{
            "filter":{ "terms":{ "family_name":"Brown" } } //filter_1
        }
    },
    "aggs":{
        "young_age":{
            "filter":{
                "range":{ "lt":40, "gt":18 } //filter_2
            },
            "aggs":{
                "age":{
                    "terms":{
                        "field":"age"
                    }
                }
            }
        }
    }
}

My aggregation "young_age" will be filtered by both filter_1 and filter_2. I don't want my aggregation to be filtered by filter_1.

As I was looking into the documentation, I thought global aggregation would solve my problem, and I wrote that query:

{
    "query":{
        "filtered":{
            "filter":{ "terms":{ "family_name":"Brown" } } //filter_1
        }
    },
    "aggs":{
        "young_age":{
            "global":{}, //<----------- add global
            "filter":{
                "range":{ "lt":40, "gt":18 } //filter_2
            },
            "aggs":{
                "age":{
                    "terms":{
                        "field":"age"
                    }
                }
            }
        }
    }
}

But then elastic search complains about my filter_2:

""" Found two aggregation type definitions [age] in [global] and [filter] """

And of course if I remove the filter_2:

{
    "query":{
        "filtered":{
            "filter":{
                "terms":{
                    "family_name":"Brown"
                }
            }
        }
    },
    "aggs":{
        "young_age":{
            "global":{},
            "aggs":{
                "age":{
                    "terms":{
                        "field":"age"
                    }
                }
            }
        }
    }
}

Then my aggregation won't be filtered by filter_1 (as expected).

So how am I suppose to apply filter_2 to my global aggregation? Or how am I supposed to achieved that? I remember writing something similar with the facet filters...

like image 347
adrienbourgeois Avatar asked Jul 28 '15 08:07

adrienbourgeois


People also ask

What is filtered aggregate?

Filter aggregationeditA single bucket aggregation that narrows the set of documents to those that match a query. The previous example calculates the average price of all sales as well as the average price of all T-shirt sales.

Is Elasticsearch good for aggregation?

Elasticsearch Aggregations provide you with the ability to group and perform calculations and statistics (such as sums and averages) on your data by using a simple search query. An aggregation can be viewed as a working unit that builds analytical information across a set of documents.


1 Answers

In my opinion this is the typical use case of a post_filter. As the doc says:

The post_filter is applied to the search hits at the very end of a search request, after aggregations have already been calculated

Your query will look like:

{
    "post_filter":{
       "terms":{
            "family_name":"Brown" //filter_1
        }
    },
   "aggs":{
        "young_age":{
            "filter":{
                "range":{ "lt":40, "gt":18 } //filter_2
            },
            "aggs":{
                "age":{
                    "terms":{
                        "field":"age"
                    }
                }
            }
        }
    }
}

In this case the search hits are all the documents in the index. Then the aggregation is calculated (before filter_1). And after that the post_filter with the filter_1 will be executed.

Edit: As you said in your commend you have many aggregations and only one that shouldn't be affected by filter_1 I fixed your query using global aggregation

{
  "query": {
    "filtered": {
      "filter": {
        "term": {
          "family_name": "Brown"
        }
      }
    }
  },
  "aggs": {
    "young_age": {
      "global": {},
      "aggs": {
        "filter2": {
          "filter": {
            "range": {
              "lt": 40,
              "gt": 18
            }
          },
          "aggs": {
            "age": {
              "terms": {
                "field": "age"
              }
            }
          }
        }
      }
    }
  }
}
like image 134
moliware Avatar answered Oct 06 '22 18:10

moliware