Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Mirrormaker 2: different partition count

I am using MM2 to replicate clusters. The current topics are overprovisioned (too many partitions) and I need to reduce them as part of the migration to fit actual needs & confluent cloud restrictions.

Is this at all possible? I see that this helps with offset replication. Is it maybe possible to replicate without offset replication to allow replicating into a reduced number of partitions?

like image 662
V-Lamp Avatar asked Mar 19 '26 19:03

V-Lamp


1 Answers

We faced the same issue and our solution is to use MirrorMaker1(kafka-mirror-maker.sh).
MirrorMaker1 uses Consumer/Producer API for replication and doesn't require partition aligning of source/target topics instead of MM2.
The documentation of fresh Kafka versions 2.8+ doesn't mention MM1, I found docs about it in previous versions of Kafka documentation, e.g. apache kafka 2.0.

like image 172
Mikhail Dubrovin Avatar answered Mar 22 '26 17:03

Mikhail Dubrovin



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!