Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Drop log line containing hash character

In my Logstash shipper I want to filter out lines commented with the hash character:

#This log row should be dropped.
But one this should not.

I was able to use grep filter, but as it is discouraged (going to be decommissioned), I'm trying to get a grok filter to do it instead. This filter is not working:

grok {
  match => ["message", "^#.*"]
  drop_if_match => true
}

I also tried placing the regex in a custom pattern file, but didn't help. Any ideas?

like image 546
Jonas Byström Avatar asked Nov 26 '13 11:11

Jonas Byström


2 Answers

Even simpler, if you're interested:

filter {
    if ([message] =~ /^#/) {
        drop{}
    }
}

The last few versions of Logstash have been putting more emphasis on branching logic directly in the config files. Takes a little getting used to, but pretty handy once you do.

like image 89
rutter Avatar answered Oct 05 '22 23:10

rutter


The correct answer is that there is a bug in drop_if_match=>true (Logstash v1.2.2). Use this type of workaround:

grok {
  ...
  add_tag => ["some_comment"]
}
if "some_comment" in [tags] {
  drop {}
}
like image 43
Jonas Byström Avatar answered Oct 05 '22 23:10

Jonas Byström