Kafka cluster with single broker

Question

I'm looking to start using Kafka for a system and I'm trying to cover all use cases.

Normally it would be run as a cluster of brokers running on virtual servers (replication factor 3-5). but some customers though don't care about resilience and a broker failure needing a manual reboot of the whole system is fine with them, they just care about hardware costs.

So my question is, are there any issues with using Kafka as a single broker system for small installations with low throughput?

Cheers

Answer 1

It's absolutely OK to use a single Kafka broker. Note, however, that with a single broker you won't have a highly available service meaning that when the broker fails you will have a downtime.

Your replication-factor will be limited to 1 and therefore all of the partitions of a topic will be stored on the same node.

Answer 2

For a proof-of-concept or non-critical dev work, a single node cluster works just fine. However having a cluster has multiple benefits. It's okay to go with a single node cluster if the following are not important/relevant for you.

• scalability [spreads load across multiple brokers to maintain certain throughput]
• fail-over [guards against data loss in case one/more node(s) go down]
• availability [system remains reachable and functioning even if one/more node(s) go down]