How to combine characters to create custom pattern in GROK

Question

I'm new to logstash and grok and have a question regarding a pattern.

Jul 26 09:46:37

The above content contains %{MONTH} %{MONTHDAY} %{TIME} and white spaces.

I need to know how to combine all these and create a pattern %{sample_timestamp}

Thanks!

Answer 1

Quotes from the Grok Custom Patterns Docs (RTFM):

First, you can use the Oniguruma syntax for named capture which will let you match a piece of text and save it as a field:

(?<field_name>the pattern here)

...

Alternately, you can create a custom patterns file.

• Create a directory called patterns with a file in it called extra (the file name doesn’t matter, but name it meaningfully for yourself)
• In that file, write the pattern you need as the pattern name, a space, then the regexp for that pattern.

So you could create a pattern file that contained the line:

CUST_DATE %{MONTH} %{MONTHDAY} %{TIME}

Then use the patterns_dir setting in this plugin to tell logstash where your custom patterns directory is.

 filter {
   grok {
     patterns_dir => ["./patterns"]
     match => { "message" => "%{CUST_DATE:datestamp}" }
   }
 }

Would result in the field:

 datestamp => "Jul 26 09:46:37"

Answer 2

Filter

use pattern_definitions to define your patterns

filter {
  grok {
    pattern_definitions => { "MY_DATE" => "%{MONTH} %{MONTHDAY} %{TIME}" }
    match => { "message" => "%{MY_DATE:timestamp}" }
  }
}

Result

{
  "timestamp": "Jul 26 09:46:37"
}

Tested using Logstash 6.5