Aggregate over the maximum timestamp lower than value

Question

So, we trying to aggregate all documents by color, where for each color, find the maximum timestamp as max_timestamp. Then, filter only buckets where max_timestamp is lower then now-5m. The idea here is to check if there is any color without reported document from the last 5 minutes.

Here is what we got by now:

{
  "size": 0,
    "aggs": {
      "colors_aggs": {
        "terms": {
          "field": "color",
            "size": 10
          },
          "aggs": {
            "max_timestamp": {
              "max": {
                "field": "timestamp"
              }
            },
            "aggs": {
              "filter": {
                "range": {
                  "timestamp": {
                    "lt": "now-5m"
                  }
                }
              }
            }
          }
        }
      }
    }

It seems to ignore the third aggregation. Buckets with timestamp greater than now-5m are shown.

Any help?

Answer 1

Perhaps you can use script to filter-out the unwanted records (in your case records with timestamp > "now - 5m") within the final aggregation, and then your final aggregation (and eventually the output) will be based only on the wanted records. The query should be something like this:

{
  "size": 0,
  "aggs": {
    "colors_aggs": {
      "terms": {
        "field": "color",
        "size": 10
      },
      "aggs": {
        "maximals": {
          "max": {
            "field": "timestamp":
          }
        },
        "max_bucket_filter": {
          "bucket_selector": {
            "buckets_path": {
              "maxs": "max_timestamp"
            },
            "script": {
              "lang": "expression",
              "script": "maxs < [CurrentUnixTime x 1000]"
            }
          }
        }
      }
    }
  }
}

Notice that the above script cannot accept the keyword now-5m so you will have to set the current unix time every time you execute the query.