Nested query in nested, filter aggregation fails

Question

I am trying to use a nested query filter inside of a nested, filter aggregation. When I do so, the aggregation returns with no items. If I change the query to just a plain old match_all filter, I do get items back in the bucket.

Here is a simplified version of the mapping I'm working with:

"player": {
  "properties": {
    "rating": {
      "type": "float"
    },
    "playerYears": {
      "type": "nested",
      "properties": {
        "schoolsOfInterest": {
          "type": "nested",
          "properties": {
            "name": {
                "type": "string",
                "index": "not_analyzed"
            }
          }
        }
      }
    }
  }
}

This query, with a match_all filter on the aggregation:

GET /players/_search
{
  "size": 0,
  "aggs": {
    "rating": {
      "nested": {
        "path": "playerYears"
      },
      "aggs": {
        "rating-filtered": {
          "filter": {
                "match_all": {}
          },
          "aggs": {
            "rating": {
              "histogram": {
                "field": "playerYears.rating",
                "interval": 1
              }
            }
          }
        }
      }
    }
  },
  "query": {
    "filtered": {
      "filter": {
        "match_all": {}
      }
    }
  }
}

returns the following:

{
   "took": 16,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 167316,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "rating": {
         "doc_count": 363550,
         "rating-filtered": {
            "doc_count": 363550,
            "rating": {
               "buckets": [
                  {
                     "key_as_string": "-1",
                     "key": -1,
                     "doc_count": 20978
                  },
                  {
                     "key_as_string": "0",
                     "key": 0,
                     "doc_count": 312374
                  },
                  {
                     "key_as_string": "1",
                     "key": 1,
                     "doc_count": 1162
                  },
                  {
                     "key_as_string": "2",
                     "key": 2,
                     "doc_count": 12104
                  },
                  {
                     "key_as_string": "3",
                     "key": 3,
                     "doc_count": 9558
                  },
                  {
                     "key_as_string": "4",
                     "key": 4,
                     "doc_count": 5549
                  },
                  {
                     "key_as_string": "5",
                     "key": 5,
                     "doc_count": 1825
                  }
               ]
            }
         }
      }
   }
}

But this query, which has a nested filter in the aggregation, returns an empty bucket:

GET /players/_search
{
  "size": 0,
  "aggs": {
    "rating": {
      "nested": {
        "path": "playerYears"
      },
      "aggs": {
        "rating-filtered": {
          "filter": {
              "nested": {
                "query": {
                  "match_all": {}
                },
                "path": "playerYears.schoolsOfInterest"
            }
          },
          "aggs": {
            "rating": {
              "histogram": {
                "field": "playerYears.rating",
                "interval": 1
              }
            }
          }
        }
      }
    }
  },
  "query": {
    "filtered": {
      "filter": {
        "match_all": {}
      }
    }
  }
}

the empty bucket:

{
   "took": 8,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 167316,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "rating": {
         "doc_count": 363550,
         "rating-filtered": {
            "doc_count": 0,
            "rating": {
               "buckets": []
            }
         }
      }
   }
}

Is it possible to use nested filters inside of nested, filtered aggregations? Is there a known bug in elasticsearch about this? The nested filter works fine in the query context of the search, and it works fine if I don't use a nested aggregation.

Answer 1

Based on the information provided, and a few assumptions, I would like to provide two suggestions. I hope it helps solve your problem.

Case 1: using reverse nested aggregation:

{
  "size": 0,
  "query": {
    "match_all": {}
  },
  "aggs": {
    "rating": {
      "nested": {
        "path": "playerYears.schoolsOfInterest"
      },
      "aggs": {
        "rating-filtered": {
          "filter": {
            "match_all": {}
          },
          "aggs": {
            "rating_nested": {
              "reverse_nested": {},
              "aggs": {
                "rating": {
                  "histogram": {
                    "field": "rating",
                    "interval": 1
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Case 2: changes to filtered aggregation:

{
  "size": 0,
  "aggs": {
    "rating-filtered": {
      "filter": {
        "nested": {
          "query": {
            "match_all": {}
          },
          "path": "playerYears.schoolsOfInterest"
        }
      },
      "aggs": {
        "rating": {
          "histogram": {
            "field": "playerYears.rating",
            "interval": 1
          }
        }
      }
    }
  },
  "query": {
    "filtered": {
      "filter": {
        "match_all": {}
      }
    }
  }
}

I would suggest you to use case 1 and verify your required results.