Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parquet Output From Kafka Connect to S3

I see Kafka Connect can write to S3 in Avro or JSON formats. But there is no Parquet support. How hard would this be to add?

like image 586
clay Avatar asked May 09 '17 19:05

clay


2 Answers

Starting with Confluent 5.4.0, there is official support for Parquet output to S3.

like image 92
clay Avatar answered Sep 30 '22 13:09

clay


The Qubole connector supports writing out parquet - https://github.com/qubole/streamx

like image 39
diomedes01 Avatar answered Sep 30 '22 14:09

diomedes01



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!