I am using Logstash and one of my applications is sending me fields like:
[message][UrlVisited]
[message][TotalDuration]
[message][AccountsProcessed]
I'd like to be able to collapse these fields, removing the top level message altogether. So the above fields will become:
[UrlVisited]
[TotalDuration]
[AccountsProcessed]
Is there a way to do this in Logstash?
Assuming the names of all such subfields are known in advance you can use the mutate filter:
filter {
mutate {
rename => ["[message][UrlVisited]", "UrlVisited"]
}
mutate {
rename => ["[message][TotalDuration]", "TotalDuration"]
}
mutate {
rename => ["[message][AccountsProcessed]", "AccountsProcessed"]
}
mutate {
remove_field => ["message"]
}
}
Alternatively, use a ruby filter (which works even if you don't know the field names):
filter {
ruby {
code => "
event.get('message').each {|k, v|
event.set(k, v)
}
event.remove('message')
"
}
}
This example works on Logstash 2.4 and later. For earlier versions use event['message'].each ...
and event[k] = v
instead.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With