How to split a JSON array inside an object

Question

I have a JSON-Message with an array in an array. I want to split that into multiple events:

{
"type": "monitor",
"server": "10.111.222.333",
"host": "abc.de",
"bean": [{
    "name": "beanName1",
    "reseted": "2015-06-05T15:10:00.192Z",
    "method": [{
      "name": "getAllXY",
      "count": 5,
      "min": 3,
      "max": 5
    },
    {
      "name": "getName",
      "count": 4,
      "min": 2,
      "max": 4
    }]
  },
  {
    "name": "beanName2",
    "reseted": "2015-06-05T15:10:00.231Z",
    "method": [{
      "name": "getProperty",
      "count": 4,
      "min": 3,
      "max": 3
    }]
  },
  {
    "name": "beanName3",
    "reseted": "2015-06-05T15:10:00.231Z"
  }]
}

Using a filter to split "bean":

input {
  stdin {
    codec => "json"
  }
}

filter {
  split {
    field => "bean"
  }
}

output {
  stdout{codec => "json"}
}

is working well:

{"type":"monitor",
 "server":"10.111.222.333",
 "host":"abc.de",
 "bean":{
   "name":"beanName1",
   "reseted":"2015-06-05T15:10:00.192Z",
   "method":[{
     "name":"getAllXY",
     "count":5,
     "min":3,
     "max":5 
    },{
     "name":"getName",
     "count":4,
     "min":2,
     "max":4
    }]},
 "@version":"1",
 "@timestamp":"2015-07-14T09:21:18.326Z"
}

{"type":"monitor",
 "server":"10.111.222.333",
 "host":"abc.de",
 "bean":{
   "name":"beanName2",
   "reseted":"2015-06-05T15:10:00.231Z",
   "method":[{
     "name":"getProperty",
     "count":4,
     "min":3,
     "max":3
   }]},
 "@version":"1",
 "@timestamp":"2015-07-14T09:21:18.326Z"
}

    ...

To seperate also the "methods", I added another split-filter:

  split {
    field => "bean"
  }
  split {
    field => "bean.method"
  }

But that way I get only an error message:

Exception in filterworker {"exception"=>#LogStash::ConfigurationError: Only String and Array types are splittable. field:bean.method is of type = NilClass

I can't access the array "method" inside the object "bean". I tried different notations with no luck. Is it possible to access the array, maybe it isn't supported yet?

Answer 1

The following code should do what you want and return one event for each method:

filter {
    if !("splitted_beans" in [tags]) {
        json {
            source => "message"
        }
        split { 
            field => "bean"
            add_tag => ["splitted_beans"]
        }
    }

    if ( "splitted_beans" in [tags] and [bean][method] ) {
        split {
            field => "bean[method]"
        }
    }
}

The second condition checks if the first method was successful and if a method exists inside your bean. So it works for beans without methods as well.