Delete documents older than 30 days in elasticsearch [closed]

Question

I want to delete documents in my elasticsearch index which are older than 30 days.

Any ideas?

EDIT:

I want this to happen automatically - no document in my index shoudl be older than 30 days. So, in my opinion there are 2 options: either using curator or DELETE requests.

I have tried both, but i failed. Somehow i have to create a filter which filters all documents older than 30 days and deletes them, when i am using DELETE http statement.

I tried with curator, but curator (as far as i understood this) deletes only whole indices. When attempting to delete indices older than 30 days with curator, my timestamp causing errors.My moment.js pattern looks like this"MMMM Do YYYY, HH:mm:ss.SSS".

EDIT 2: I added the following to my logstash configuration:

elasticsearch
    {
    hosts => ["http://localhost:9200"]
    index => "logstash-%{type}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
    }

So logstash creates for every type and every day a particular index. Now i can use curator to delete the indices older than a specific date.

Problem solved imho.

Answer 1

You can use DELETE query for that: https://www.elastic.co/guide/en/elasticsearch/reference/1.6/docs-delete-by-query.html in example the query will delete everything older than: 2016-02-29

DELETE index_name/_query
{
  "query": {
    "filtered": {
      "query": {
        "query_string": {
          "query": "*"
        }
      },
      "filter": {
        "range": {
          "@timestamp": {
            "lte": "2016-02-29"
          }
        }
      }
    }
  }
}

Update >6.4

According to the official documentation, this function has been deprecated and replaced by _delete_by_query

POST index_name/_delete_by_query
{
  "query": { 
    "match": {
      "message": "some message"
    }
  }
}

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html