I'm trying to split a JSON array into multiple events. Here's a sample input:
{"results" : [{"id": "a1", "name": "hello"}, {"id": "a2", "name": "logstash"}]}
Here's my filter and output config:
filter {
split {
field => "results"
}
}
stdout {
codec => "rubydebug"
}
This produces 2 events, one for each of the JSONs in the array. And it's close to what I'm looking for:
{
"results" => {
"id" => "a1",
"name" => "hello"
},
"@version" => "1",
"@timestamp" => "2015-05-30T18:33:21.527Z",
"host" => "laptop",
}
{
"results" => {
"id" => "a2",
"name" => "logstash"
},
"@version" => "1",
"@timestamp" => "2015-05-30T18:33:21.527Z",
"host" => "laptop",
}
The problem is the nested "results" part. "results" being the default value for the target parameter. Is there a way to use the split filter without producing the nested JSON, and get something like this:
{
"id" => "a1",
"name" => "hello"
"@version" => "1",
"@timestamp" => "2015-05-30T18:33:21.527Z",
"host" => "laptop",
}
{
"id" => "a2",
"name" => "logstash"
"@version" => "1",
"@timestamp" => "2015-05-30T18:33:21.527Z",
"host" => "laptop",
}
The purpose is to feed this to the ElasticSearch output with each event being a document with document_id => "id". Any good solutions are welcomed!
If you know what all of the fields will be (as it appears you do), you can simply rename the fields:
mutate {
rename => [
"[results][id]", "id",
"[results][name]", "name"
]
remove_field => "results"
}
If you didn't know what all of the fields were, you could write a ruby
code filter that did a event['results'].each...
and created new fields from the sub-fields of results.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With