How to write a custom flume OG sink

Question

We're using flume and I need to collect some log messages into rabbitmq. I found a source implementation that reads messages from rabbitmq, but I couldn't find a sink that can write messages into rabbit. So I was thinking about writing one myself. Looking at sample implementations like logsandra made me think it shouldn't be too difficult.

However I couldn't find any documentation on how to write a custom sink. I didn't find a maven repo for the flume jars, or setup instructions on how to deploy a custom sink.

Can anyone share his experience, or better, point me to an existing tutorial.

Answer 1

Eventually we wrote one ourselves, and open sourced it - https://github.com/kenshoo/flume-rabbitmq-sink

there's a cloudera repo now https://ccp.cloudera.com/display/CDHDOC/Using+the+CDH3+Maven+Repository (there wasn't when I asked)

And flume user guide now explains well how to configure a custom plugin

http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_extending_via_sink_source_decorator_plugins