Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

writing grok pattern to split key value pairs

This is my first experience with using grok and elasticsearch. I'm trying to write a grok file to do parse the following line.

2017-07-25 10:48:23,477 [[api-my-app-v1-20.0.0].apiHttpListenerConfig.worker.58] INFO  esb.api-my-app-v1.get-blah.http.response - transactionID=f61b8053-70d2-11e7-b274-3639cc5335d0 creationTime=2017-07-25T10:48:23.254+10:00 txnState=END timeTaken=11

So far I have written this grok...

%{TIMESTAMP_ISO8601:timestamp}\s+%{DATA:thread}\s+%{LOGLEVEL:loglevel}\s+%{JAVACLASS:category}\s+-\s+%{GREEDYDATA:msgbody}

It gives me back this....

{
  "timestamp": [
    [
      "2017-07-25 10:48:23,477"
    ]
  ],
  "YEAR": [
    [
      "2017"
    ]
  ],
  "MONTHNUM": [
    [
      "07"
    ]
  ],
  "MONTHDAY": [
    [
      "25"
    ]
  ],
  "HOUR": [
    [
      "10",
      null
    ]
  ],
  "MINUTE": [
    [
      "48",
      null
    ]
  ],
  "SECOND": [
    [
      "23,477"
    ]
  ],
  "ISO8601_TIMEZONE": [
    [
      null
    ]
  ],
  "thread": [
    [
      "[[api-my-app-v1-20.0.0].apiHttpListenerConfig.worker.58]"
    ]
  ],
  "loglevel": [
    [
      "INFO"
    ]
  ],
  "category": [
    [
      "esb.api-my-app-v1.get-blah.http.response"
    ]
  ],
  "msgbody": [
    [
      "transactionID=f61b8053-70d2-11e7-b274-3639cc5335d0 creationTime=2017-07-25T10:48:23.254+10:00 txnState=END timeTaken=11"
    ]
  ]
}

This is almost what I want. How can I split the msgbody from my current result into key value pairs?

thanks

like image 741
Richie Avatar asked Jul 25 '17 05:07

Richie


1 Answers

With the kv filter:

kv { 
   source => "msgbody" 
}

you'll have the key-pair values from the msgbody in fields in your result. Also you won't have to change your grok pattern if the keys change.

like image 153
baudsp Avatar answered Oct 07 '22 13:10

baudsp