How to assign a specific kafka consumer to a specific partition

Question

I have a "topic" with 2 partitions.

topic-0
topic-1

And I have 2 input message channels

input0
input1

with same destination "topic" and same group.

My understanding is that, since it is a same group each topic will be automatically assigned a partition.

Is possible to assign a specific partition to a specific message channel?

Answer 1

You can assign specific partitions to Kafka consumer with consumer's auto-rebalance disabled along with the appropriate instanceCount and instanceIndex properties.

For instance, in your case,

--spring.cloud.stream.bindings.input0.consumer.instanceCount=2 --spring.cloud.stream.bindings.input1.consumer.instanceCount=2 --spring.cloud.stream.bindings.input0.consumer.instanceIndex=0 --spring.cloud.stream.bindings.input1.consumer.instanceIndex=1 --spring.cloud.stream.bindings.input0.group=mygroup --spring.cloud.stream.bindings.input1.group=mygroup --spring.cloud.stream.kafka.bindings.input0.consumer.autoRebalanceEnabled=false --spring.cloud.stream.kafka.bindings.input1.consumer.autoRebalanceEnabled=false --spring.cloud.stream.bindings.input0.destination=topic --spring.cloud.stream.bindings.input1.destination=topic

The above configuration will assign the topic's partitions for each consumer (when binding the input channels) based on the modulo using the partition , instanceCount and instanceIndex values.