Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Logstash--Dynamic indexes on the basis of fields set in logstash forwarder

Tags:

logstash

I want to create indexes from values of 'Application' field which I set in logstash-forwarder on client machine. For example: in logstash-forwarder.conf on CRM client, I write

{
"paths": [
"/var/log/crm/crmERROR.log"
],
"fields": {**"**Application**":"CRM"**,"Sub-System":"Mysystem", "type":"Error-logs "
}

in LSF config on sales client 2 I say 
{
"paths": [
"/var/log/sales/SalesERROR.log"
],
"fields": {**"Application":"Sales"**,"Sub-System":"Myststem", "type":"Error-logs "
}

all those logs will be sent to redis through LS-shipper. LS-indexer will pick it up from redis. Now in LS-Indexer, I want to create indexes on the basis of values of field "Application" (CRM,Sales). How Can I get these values of Application field so that I can use it in output to create indexname.

br, Sunil.

like image 239
sunil chaudhari Avatar asked Feb 09 '23 15:02

sunil chaudhari


1 Answers

The elasticsearch{} output allow you to specify the index name, e.g.:

output {
    elasticsearch {
        index => "logstash-%{+YYYY.MM.dd}"
    }
}

You can use variables in the index name:

index => "%{my_field}-%{+YYYY.MM.dd}"
like image 51
Alain Collins Avatar answered Feb 12 '23 14:02

Alain Collins