logstash: multiple logfiles with different pattern

Question

We want to set up a server for logstash for a couple of different project in our company. Now I try to enable them in Kibana. My question is: If I have different patterns of the logfiles, how can I build for them a filter? example: logstash.conf:

input {
  file {
    type => "A"
    path => "/home/logstash/A/*"
    start_position => "beginning"
  }
 file {
    type => "B"
    path => "/home/logstash/B*"
    start_position => "beginning"
  }
}

filter {
  multiline {
      pattern => "^%{TIMESTAMP_ISO8601}"
      negate => true
      what => "previous"
  }
  grok {
     type => A
     match => [ "message", "%{TIMESTAMP_ISO8601:logdate} %{DATA:thread %{LOGLEVEL:level}\s*%{DATA:logger_name}\s*-\s*%{GREEDYDATA:log_text}"]

    add_tag => [ "level_%{level}" ]
  }
  date {
        match => ["logdate", "YYYY-MM-dd HH:mm:ss,SSS"]
  }
  grok {
        type => B
        match => [ any other pattern ... 
 }
}
output {
  elasticsearch { embedded => true }
}

do I have to create for each project (A,B,C,...) an own filter, and what do I have to do, when I have for each project again different pattern of the logfiles?

Answer 1

You only need to create a filter for all projects.

For Logstash 1.3.3, You can use if statement to distinct each project grok. For example,

filter {

   multiline {
       pattern => "^%{TIMESTAMP_ISO8601}"
       negate => true
       what => "previous"
   }

   if [type] == "A"  {
      grok {
          match => [ any other pattern ... 
      }
   }
   else if [type] == "B" {
      grok {
          match => [ any other pattern ... 
      }
   }
}

Hope this can help you.