ELK: Setup multiple http inputs of logstash ELK stack

Question

Question:

• How to setup multiple http inputs of logstash ELK stack

What I already have:

• Working ELK docker image based on: https://github.com/deviantony/docker-elk
• Working logstash-input-http-plugin based on: https://www.elastic.co/blog/introducing-logstash-input-http-plugin
• My logstash.conf file looks like:

input {
  http {
        host => "0.0.0.0"
        port => "5000"
    }
}

output {
  elasticsearch {
      hosts => "elasticsearch:9200"
  }
}

• And I can easly send my Component logs (as JSON) using postman on URL: http://localhost:5000

What I need:

• Multiple http inputs because I have multiple Components - something like (but second input does not listen to requests):

input {
  http {
        host => "0.0.0.0"
        port => "5000"
  }
  http {
      host => "0.0.0.0"
      port => "7070"
  }
}

• I have to distinguish those Components in Kibona

Answer 1

You can set a type for each input and use that type to generate the index name:

input {
  http {
    host => "0.0.0.0"
    port => "5000"
    type => "A"
  }

  http {
    host => "0.0.0.0"
    port => "5001"
    type => "B"
  }
}

Using the type may suffice, as you can filter the records using it. But you may also need to store each type of record in a different index since each type may use a different type for the same field. This causes a mapping conflict.

output {
  elasticsearch {
    hosts => "elasticsearch:9200"
    index => "%{[type]}-%{+YYYY.MM.dd}"
  }
}