Kibana + Logstash + Elastic group exceptions by stacktrace

Question

For now I have logstashed installed and I can see all [ERROR] messages and stacktraces in Kibana.

However I would like to group my exceptions by its stacktraces to see only unique exceptions. For example, I have 2 different NullPointerException throwned from different classes/lines of codes and 1 IllegalArgumentException. This exceptions repeats in logs multiple times, and I can see all occurrence in Kibana.

What I would like to see, something similar to this

1. NullPointerException + trace - thwwn 78 times.
2. Another NullPointerException + trace - thrown 112 times.
3. IllegalArgumentException + trace - thrown 991 times.

This is how fabric.io works for mobile crashes/issues. It is possible to do something similar ?

Answer 1

The way we did this was to set up a data table visualisation with a Terms query on the field containing the stacktrace. Mind you however to aggregate on the .raw version of your field otherwise you will see the analysed (e.g. split) version of the stacktraces which is not what you want.

The second problem I came across is that there is a maximum size of the text to be used for the terms filter (unfortunately I can't find the docs for that). Anything larger than that is simply omitted from the aggregation. I worked around that by creating an additional field containing the first 200 characters of the stacktrace which I then use to aggregate on.

    grok {
        match => [ "exceptionTxt","(?<exceptionTxtShort>^.{0,200})"]
    }

It's not perfect but it does the trick for us. If anyone knows the link to the docs and/or how to properly work around this limitation please comment.