It is possible to automatically delete data older than for instance 10 days from elasticsearch in real-time?

Question

curl -XGET '127.0.0.1:9200/messages/message/_search?pretty' returns data like shown below. I wonder whether it is possible to automatically delete data older than for instance 10 days from elasticsearch preferably in real time? I added my example data because there is a field date that could be used in this case. Or maybe there is a different more recommended method?

{
  "took" : 22,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "messages",
        "_type" : "message",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "message" : "example message1"
        }
      },
      {
        "_index" : "messages",
        "_type" : "message",
        "_id" : "ZODslt0LZ1T6GMrC",
        "_score" : 1.0,
        "_source" : {
          "date" : "2018-05-25T10:06:06Z",
          "message" : "example message1"
        }
      }
    ]
  }
}

Answer 1

Elastic Curator is exactly what you are looking for. You should create a separate file for an index for each day.

For example if your index has pattern like that: YOUR_INDEX_NAME-%{+YYYY.MM.dd} then you should apply configuration below:

actions:
  1:
    action: delete_indices
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: YOUR_INDEX_NAME-
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'  <--- date pattern in your index name
      unit: days
      unit_count: 10          <--- after how many days delete the index