Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Make logstash add different inputs to different indices

I have setup logstash to use an embedded elastisearch.
I can log events.
My logstash conf looks thus:
https://gist.github.com/khebbie/42d72d212cf3727a03a0

Now I would like to add another udp input and have that input be indexed in another index.

Is that somehow possible? I would do it to make reporting easier, so I could have system log events in one index, and business log events in another index.

like image 956
khebbie Avatar asked Nov 26 '14 09:11

khebbie


People also ask

Can Logstash have multiple inputs?

Only use input once.

Can Logstash have multiple outputs?

Using Logstash multiple outputs Furthermore, we can forward the filtered data of Logstash either to a single output destination or multiple outputs by filtering the inputs in a specific manner, resulting in the outputs being distributed to that particular stream for each of the inputs received.

Does Logstash create index in Elasticsearch?

Logstash does not create index on elasticsearch.

How do I change the index name in Logstash?

Tip: To edit your Logstash filters for any Stack choose View Stack Settings > Logstash Pipelines from your Dashboard. Alternatively, inside your condition you can specify the index name using add_field .


1 Answers

Use an if conditional in your output section, based on e.g. the message type or whatever message field is significant to the choice of index.

input {
  udp {
    ...
    type => "foo"
  }
  file {
    ...
    type => "bar"
  }
}

output {
  if [type] == "foo" {
    elasticsearch {
      ...
      index => "foo-index"
    }
  } else {
    elasticsearch {
      ...
      index => "bar-index"
    }
  }
}

Or, if the message type can go straight into the index name you can have a single output declaration:

elasticsearch {
  ...
  index => "%{type}-index"
}
like image 139
Magnus Bäck Avatar answered Oct 06 '22 11:10

Magnus Bäck