Filebeat > Logstash > ElasticSearch - Lumberjack Error

Question

Trying to get Filebeat to work with logstash. Currently I am getting this error:

2016/11/14 04:54:27.721478 output.go:109: DBG  output worker: publish 2047 events
2016/11/14 04:54:27.756650 sync.go:85: ERR Failed to publish events caused by: lumberjack protocol error
2016/11/14 04:54:27.756676 single.go:91: INFO Error publishing events (retrying): lumberjack protocol error
2016/11/14 04:54:28.782729 sync.go:85: ERR Failed to publish events caused by: lumberjack protocol error
2016/11/14 04:54:28.782756 single.go:91: INFO Error publishing events (retrying): lumberjack protocol error
2016/11/14 04:54:30.786983 sync.go:85: ERR Failed to publish events caused by: lumberjack protocol error
2016/11/14 04:54:30.787017 single.go:91: INFO Error publishing events (retrying): lumberjack protocol error
2016/11/14 04:54:34.808528 sync.go:85: ERR Failed to publish events caused by: lumberjack protocol error
2016/11/14 04:54:34.808578 single.go:91: INFO Error publishing events (retrying): lumberjack protocol error

Does anyone have a solution or perhaps you can point me in the right direction?

Filebeat v5 and logstash v5.

doing this locally.

Thanks!

Answer 1

It could be because your logstash is not visible to your filebeat. Please start your logstash on beat as input. I have started my logstash on port no 31311.

    input {
  beats {
    host => "0.0.0.0"
    port => 31311
  }
}

Configuration in filebeat.yml : ElasaticSearch output is by default. Please comment it and uncomment logstash output configuration. It should look like as below.

output.logstash:
  # The Logstash hosts
  hosts: ["localhost:31311"]

Please take care if you have any authentication set up between these two. I am able to see my messages into logstash and finally in Kibana after doing this configuration. Hope it will work.