How to control and rotate storage data at elasticsearch

Question

Could you help me ? I want to use logstash + elasticsearch + kibana for userfriendly mapping logs. Explain me please. I can't understand where does elasticsearch store data (for example my logs) and how I can rotate it?

Answer 1

Elasticsearch stores it's data in a way that you can't modify outside of the Elasticsearch API. Data that goes in from logstash typically goes into a date-stamped index (ie logstash-%{+YYYY.MM.dd}).

You can use Elasticsearch Curator to have it delete indexes older than a certain number of days. The documentation shows this example:

curator --host localhost delete indices --older-than 30 --time-unit days --timestring '%Y.%m.%d'

---

For curoator with version 5.7.6, you should run command with the Singleton Command Line Interface mode as below:

curator_cli --host elasticsearch delete_indices --filter_list '{"filtertype":"age","source":"name","direction": "older","timestring":"%Y.%m.%d","unit":"days","unit_
count":30}'

ref: https://www.elastic.co/guide/en/elasticsearch/client/curator/5.7/singleton-cli.html

which would connect to Elasticsearch on the current node and delete any index older than 30 days.