what is the regexp pattern for multiline (logstash)

Question

Currently I have:

multiline {
 type => "tomcat"
 pattern => "(^.+Exception: .+)|(^\s+at .+)|(^\s+... \d+ more)|(^\s*Caused by:.+)|(---)"
 what => "previous"
}

and this is part of my log:

TP-xxxxxxxxxxxxxxxxxxxxxxxx: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
    at xxxxxx
Caused by: xxxxxxxxx
    at xxxxxx
Caused by: xxxxxxxxx   
--- The error occurred in xxxxxxxxx.  
--- The error occurred xxxxxxxxxx.  

My pattern doesn't work here. Probably because i added the (---) at the end. What is the correct regexp to also add the --- lines?

Thanks

Answer 1

You'll want to account for the other characters on the line as well:

(^---.*$)

Answer 2

I have put your regex and text into these online regex buddies and tried the suggestion of Eric:

• http://www.regextester.com/
• http://www.regexr.com/

Sometimes these online buddies really help to clear the mind. This picture shows what is recognized:

If I were stuck on this, I wouldn't focus on the regex itself any further. Rather I'd check these points:

• As there are different regex dialects, what dialect is used by logstash? What does it mean to my pattern?
• Are there any logstash specific modifiers that are not set and need to be set?
• As Ben mentioned, there are further filter tools. Would it help to use grok instead?

Answer 3

If one log event start with a timestamp or a specific word, for example, in your logs if all logs start with TP, then you can use it as filter pattern.

    multiline {
            pattern => "^TP"
            what => "previous"
            negate => true
    }

With this filter you can multiline your logs easy, no need to use complex patterns.