Fluentd Elasticsearch target index

Question

I'm using Fluentd to transfer the data into Elasticsearch.

td-agent.conf

## ElasticSearch
<match es.**>
  type elasticsearch
  target_index_key @target_index  
  logstash_format true
  flush_interval 5s
</match>

Elasticsearch index :

"logstash-2016.02.24" : {
    "aliases" : { },
    "mappings" : {
      "fluentd" : {
        "dynamic" : "strict",
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          "dummy" : {
            "type" : "string"
          }

        }
      }
    },

Transmit json data :

$ curl -X POST -d 'json={"@target_index": "logstash-2016.02.24","dummy":"test"}' http://localhost:8888/es.test

It should write the data to the given index instead of that It creates new index - logstash-2016.02.25 and it will write data into that. I want to write data to the given index.

Here is the Fluentd elasticsearch github link : https://github.com/uken/fluent-plugin-elasticsearch

Please correct me if I'm missing something.

Answer 1

try this, its due to logstash_format true, please enter your index name in below index_name field (default value is fluentd)

<match es.**>
@type elasticsearch
host localhost
port 9200
index_name <.....your_index_name_here.....>
type_name fluentd
flush_interval 5s
</match>

after run this, please check index created or not by load below url in your browser

http://localhost:9200/_plugin/head/

have a good luck