Menu
I have an application in production that has to process several gigabytes of messages per day. I like the Kafka architecture and performance a lot; it perfectly fits my needs.
I'd like to replace my messaging layer with Kafka at some point. Is the 0.7.1 version good enough for production use in terms of stability and consistency in performance?
741
KRaft is intended to be pronounced like craft (as in craftsmanship ). It is currently PREVIEW AND SHOULD NOT BE USED IN PRODUCTION, but it is available for testing in the Kafka 3.1 release. When the Kafka cluster is in KRaft mode, it does not store its metadata in ZooKeeper.
Presently, Kafka developers are working on full-feature parity between KRaft and ZooKeeper, which is said to be closing in. KRaft mode actually has been available since Kafka 2.8, released in April 2021, but not in production-ready status; Kafka 3.3 will be the first production-ready release.
Kafka is real-time! Kafka provides capabilities to process trillions of events per day. Each Kafka broker (= server) can process tens of thousands of messages per second. End-to-end latency from producer to consumer can be as low as ~10ms if the hardware and network setup are good enough.
It is definitely in use at several Big Data companies already, including LinkedIn, where it was created (and later open sourced), and Tumblr. Just Tumblr by itself handles many gigabytes of messages per day. I'm sure LinkedIn is way up there too. You can see a list of companies known to currently use it here:
https://cwiki.apache.org/confluence/display/KAFKA/Powered+By
Also, be sure to subscribe to their mailing list, there are lots of people actively trying it out and using it in production environments.
I'm sure it can handle whatever volume you can throw at it.
113
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
