can i set logstash default elasticsearch mapping through elasticsearch-template.json

Question

I use logstash + elasticsearch to collect syslog and want to set ttl for log ageing

I find a file named elasticsearch-template.json in the logstash,the path is logstash/logstash-1.4.2/lib/logstash/outputs/elasticsearch/elasticsearch-template.json

I add ttl info in the file like this:

{
  "template" : "logstash-*",
  "settings" : {
    "index.refresh_interval" : "5s"
  },
  "mappings" : {
    "_default_" : {
       "_all" : {"enabled" : true},
       "dynamic_templates" : [ {
         "string_fields" : {
           "match" : "*",
           "match_mapping_type" : "string",
           "mapping" : {
             "type" : "string", "index" : "analyzed", "omit_norms" : true,
               "fields" : {
                 "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
               }
           }
         }
       } ],
        "_ttl": {
         "enabled": true,
         "default": "1d"
       },
       "properties" : {
         "@version": { "type": "string", "index": "not_analyzed" },
         "geoip"  : {
           "type" : "object",
             "dynamic": true,
             "path": "full",
             "properties" : {
               "location" : { "type" : "geo_point" }
             }
         }
       }
    }
  }
}

then restart logstash, delete all elasticsearch index. I check the new index's mapping in the elasticsearch, but it didn't work in this way.

How can I config the index template?

Answer 1

you need to change your logstash configuration.

if you have followed the default settings, logstash has already created a template inside elasticsearch named logstash, logstash will keep on using that template stored in elasticsearch unless you tell it not to explicitly.

modify that template file you found but in addition to that, in your logstash configuration, set the following:

output {
  elasticsearch {
    ...
    template_overwrite => true
    ...
  }
}