How to reference environment variables in logstash configuration file?

Question

Is it possible to reference environment variables in logstash configuration?

In my case, i want to make my elasticsearch address configurable that i have set in the environment.

Answer 1

With logstash 2.3, you can set environment variable references into Logstash plugins configuration using ${var} or $var. https://www.elastic.co/guide/en/logstash/current/environment-variables.html

Before logstash 2.3, you can use the "environment" filter plugin which is community maintained.

Documentation at : https://www.elastic.co/guide/en/logstash/current/plugins-filters-environment.html#plugins-filters-environment-add_field_from_env

How to install this plugin:

$LOGSTASH_HOME/bin/plugin install logstash-filter-environment

Source code at : https://github.com/logstash-plugins/logstash-filter-environment

The main part is:

# encoding: utf-8
require "logstash/filters/base"
require "logstash/namespace"


# Set fields from environment variables
class LogStash::Filters::Environment < LogStash::Filters::Base
  config_name "environment"

  # Specify a hash of fields to the environment variable
  # A hash of matches of `field => environment` variable
  config :add_field_from_env, :validate => :hash, :default => {}

  public
  def register
    # Nothing
  end # def register

  public
  def filter(event)
    return unless filter?(event)
    @add_field_from_env.each do |field, env|
      event[field] = ENV[env]
    end
    filter_matched(event)
  end # def filter
end # class LogStash::Filters::Environment

Answer 2

I can hardly believe, that these are the only solutions left: Hacking logstash or using some kind of templating system to re-write the config.

Actually, I do not want to touch or tweak the config for different deployment-scenarios: All I want is to pass in some parameters to connect logstash to the outside world (e.g. where elasticsearch is located, usernames/credentials to connect to other systems). I googled for an hour now an all I could find were these awkwardly complicated solutions for this very simple and common problem.

I sincerely hope, that someone comes up with a better idea like

%{ENV[ELASTICSEARCH_HOST]}}