Logstash output to Elasticsearch on AWS EC2

Question

I'm having trouble configuring logstash to output to an Elasticsearch cluster on AWS EC2.

I'm using Logstash version 1.1.5 and Elasticsearch 1.19.8.

This is my output configuration in logstash:

output {
  stdout { debug => true debug_format => "json"}
  elasticsearch {
    cluster => "logstash-searcher"
    node_name => "logstash-indexer"
  }
}

and this is the corresponding configuration in elasticsearch.yml

cluster.name: logstash-searcher
path.data: /usr/local/elasticsearch/data
path.work: /usr/local/elasticsearch/tmp
path.logs: /usr/local/elasticsearch/logs
path.plugins: /usr/local/elasticsearch/plugins
bootstrap.mlockall: true
cloud.aws.region: eu-west-1
cloud.aws.access_key: --
cloud.aws.secret_key: --
discovery.type: ec2
discovery.ec2.host_type: public_ip
discovery.ec2.groups: elasticsearch
gateway.type: s3
gateway.s3.bucket: es-logstash

transport.tcp.port: 9300-9400

I start logstash using:

java -jar logstash-1.1.5-monolithic.jar agent -f shipper.conf

And after a while of startup I get these failures:

Failed to index an event, will retry {:exception=>org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [1m], 

My suspicion is that logstash needs to use something like the cloud-aws for its elasticsearch client to be able to find the cluster. Does anyone have an example configuration that works on aws?

Answer 1

The problem is that the embedded elasticsearch instance of logstash was using its default discovery mode. Since the elasticsearch cluster is configured with cloud-aws the embedded elasticsearch of logstash needs to as well.

To do that you have to add an elasticsearch.yml configuration file to the working directory of logstash. You also need to supply the cloud-aws plugin by adding it to the class path.

java -cp logstash-1.1.7-monolithic.jar:cloud-aws/* logstash.runner agent -f shipper.conf

Using this configuration I managed to get logstash to output to my elasticsearch cluster.