Testing custom Logstash filters

Question

We are using Ansible and have Logstash.

How do I write a few tests to cover our custom logstash filters? What I want to do is this:

1. given a Logstash config with filters,
2. pass it a log line (or multiline log record),
3. see it successfully parsed into pieces.

I know there's this – https://github.com/elastic/logstash/wiki/Tips:Testing-your-filters, but I don't understand how this is useful – it appears outdated.

Answer 1

I found this and ended up with following working test code:

# simple_filter_spec.rb
#
# run using:
#   bundle exec rspec simple_filter_spec.rb

require "logstash/devutils/rspec/spec_helper"

LogStash::Environment::LOGSTASH_HOME = `gem which logstash-core`
module LogStash::Environment
  unless self.method_defined?(:pattern_path)
    def pattern_path(path)
      ::File.join(LOGSTASH_HOME, "patterns", path)
    end
  end
end


require "logstash/filters/grok"

describe LogStash::Filters::Grok do
  config <<-CONFIG
  filter {
    grok {
      match => { "message" => "%{SYSLOGLINE}" }
      singles => true
      overwrite => [ "message" ]
    }
  }
  CONFIG

  sample "Mar 16 00:01:25 evita postfix/smtpd[1713]: connect from camomile.cloud9.net[168.100.1.3]" do
    insist { subject["tags"] }.nil?
    insist { subject["logsource"] } == "evita"
    insist { subject["timestamp"] } == "Mar 16 00:01:25"
    insist { subject["message"] } == "connect from camomile.cloud9.net[168.100.1.3]"
    insist { subject["program"] } == "postfix/smtpd"
    insist { subject["pid"] } == "1713"
  end
end

And my Gemfile looked like this:

source 'https://www.rubygems.org'

platform :jruby do
  gem 'pry'
  gem 'rspec'
  gem 'logstash-core'
  gem 'logstash-devutils'
  gem 'logstash-filter-grok'
end