Kafka to S3 - How to loading slices from kafka to S3

Question

It's not clear to me if there is some kind of plugin to consume data from kafka topics and insert them to the S3.

I already found this topic, but I could not solve this issue yet, there is this project, but honestly is hard to use because the last commit is from 2 years ago.

The main goal to me it was consume directly from kafka to spark jobs, but I think this can be a kind of complicated, so if could populate S3 with slices of events from kafka is enough to me.

Also, there is any consumer example in scala? It is kind of funny because kafka is built in scala but the documentation code is java. =p

I appreciate any help

Updated:

Camus may be a option too

Answer 1

This tool from pinterest was the perfect answer for me.

Secor