Fluentd to Logstash output plugin

Question

I am trying to read from the scribe server using flunetd and output those logs to be stored in logstash for now. I know it's very stupid to log the scribe_central logs to another central logger, but we need this to be done in our current architecture.

Does anyone know if there is any plugin to do that? I searched Google but could not find any.

Answer 1

For Scribe <-> Fluentd, there is fluent-plugin-scribe:

For Fluentd <-> Logstash, a couple of options:

1. Use Redis in the middle, and use fluent-plugin-redis and input_redis on Logstash's side. So it would be Fluentd -> Redis -> Logstash. This is what Logstash recommends anyway with log shippers + Logstash.
2. Alternatively, you can use Fluentd's out_forward plugin with Logstash's TCP input. Logstash has Fluentd codec to handle the input coming from Fluentd.

Answer 2

You can forward it directly to your logstash tcp input :)

I wrote a flunetd output plugin for forwarding fluentd events to a generic receiver through a secured tcp connection (can be configured for non-secured as well).

To add the plugin to your fluentd agent, use the following command:

gem install fluent-plugin-loomsystems

Also see full original answer, good luck.