logstash _grokparsefailure issues

Question

I'm having issues with grok parsing. In ElasticSearch/Kibana the lines I match come up with the tag _grokparsefailure.

Here is my logstash config :

input { 
    file { 
     type => logfile 
     path => ["/var/log/mylog.log"] 
    } 
  } 
filter { 
    if [type] == "logfile" 
    { 
      mutate {
      gsub => ["message","\"","'"]
      }  

    grok 
        { match => { "message" => "L %{DATE} - %{TIME}: " } } 
    } 
} 

output { 
   elasticsearch { host => localhost port => 9300 } 
}

lines/patterns I'm trying to match : L 08/02/2014 - 22:55:49: Log file closed : " finished "

I tried the debugger on http://grokdebug.herokuapp.com/ and it works fine, my pattern matches correctly.

Lines I want to parse might contain double quotes, and I've read there can be issues with the way grok handles and escapes them. So I tried to mutate to replace " with ' to avoid issues but no luck.

Any ideas ? How can I debug this ?

Thanks

Answer 1

Found out the issue, it was around double quotes.

Needed to use simple quote to define the grok filter, and escape double quotes.

match => { 'message' => 'L %{DATE:date} - %{TIME:time}: \"string_between_doublequotes\" '