Tags index with filebeat and logstash

Question

I use logstash-forwarder and logstash and create a dynamic index with tags with this configuration:

/etc/logstash/conf.d/10-output.conf

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "logstash-%{tags}-%{+YYYY.MM.dd}"
  }
}

/etc/logstash-forwarder.conf

"files": [
    {
      "paths": [
        "/var/log/httpd/ssl_access_log",
        "/var/log/httpd/ssl_error_log"
       ],
      "fields": { "type": "apache", "tags": "mytag" }
    },

The associated filebeat configuration is:

/etc/filebeat/filebeat.yml

filebeat:
  prospectors:
    -
     paths:
       - /var/log/httpd/access_log
     input_type: log
     document_type: apache
     fields:
       tags: mytag

In Kibana, instead of mytag I see beats_input_codec_plain_applied on all of my indices.

Answer 1

I can see two problems mentioned in this topic. Let me summarize for my own benefit and hopefully for other visitors struggling with that problem too.

1. format to add tag(s) in filebeat prospector (per prospector tags available since 5.0 or 1.2.3 as a-j noticed) configuration

bad:

 fields:
       tags: mytag

good:

 fields:
       tags: ["mytag"]

However, there's more important issue

2. Tags are getting concatenated. We want tags to be an array, but if we ship the newly added tags to logstash we'll see them being a concatenated strings in ES.

If you are adding only one tag, the workaround (as per hellb0y77) would be to remove the automatic tag that filebeat adds, in logstash (central server side):

filter {
    if "beats_input_codec_plain_applied" in [tags] {
        mutate {
            remove_tag => ["beats_input_codec_plain_applied"]
        }
    }
}

This would not work if one wanted to add multiple tags in filebeat.

One would have to make logstash split a concatenated string and add each item to tags. Perhaps it would be better in this case, to put tags on filebeat end into some custom field, not "tags" field and extract them from that custom field on logstash.

Anyway, there seems to be no way to make it work by changing filebeat configuration. The only way is by doing some parsing on receiving logstash filter chain. See also https://github.com/elastic/filebeat/issues/220

If you can remove logstash then this could also be solution for you. When sending logs from filebeat directly to elasticsearch, the tags appear in ES as expected.