I'm trying to match event messages with several regular expressions. I was going for the use of grep filter, but its deprecated so I'm trying for the drop with negation.
The functionality I'm looking for is to have all events dropped unless the message matches several regular expressions.
The filter bellow does not work, but tested individually both expressions work fine. What am I missing?
filter {
if ([message] !~ ' \[critical\]: ' or [message] !~ '\[crit\]: ') {
drop { }
}
}
Grok is filter within Logstash that is used to parse unstructured data into something structured and queryable. Regular expression is a sequence of characters that define a search pattern.
Put simply, grok is a way to match a line against a regular expression, map specific parts of the line into dedicated fields, and perform actions based on this mapping. Built-in, there are over 200 Logstash patterns for filtering items such as words, numbers, and dates in AWS, Bacula, Bro, Linux-Syslog and more.
A grok pattern is like a regular expression that supports aliased expressions that can be reused. This processor comes packaged with many reusable patterns. If you need help building patterns to match your logs, you will find the Grok Debugger tool quite useful! The Grok Constructor is also a useful tool.
I was reading a bit more and went along with painting the events with grok by adding a tag and dropping them in the end, if the tag was not there:
filter {
grok {
add_tag => [ "valid" ]
match => [
"message", ".+ \[critical\]: ?(.+)",
"message", ".+ \[crit\]: ?(.+) ",
"message", '.+ (Deadlock found.+) ',
"message", "(.+: Could not record email: .+) "
]
}
if "valid" not in [tags] {
drop { }
}
mutate {
remove_tag => [ "valid" ]
}
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With