Menu
I see no mention of message keys (org.apache.kafka.clients.producer.ProducerRecord.key) except that they may be used for topic partitioning.
Am I free to put whatever data I like in the key or are there some special semantics that I should conform to?
The key seems like a convenient header to put small amounts of metadata. Is this a bad idea?
314
Kafka message keys can be string values or Avro messages, depending on how your Kafka system is configured. The format of the message keys determines how message key values are stored in the record, and how you work with those values.
In Kafka, the messages are guaranteed to be processed in order only if they share the same key (and you use the default partitionner, but let's come back to that later).
Each message in a topic is assigned a sequential number that uniquely identifies the message within a partition. This number is called an offset, and is represented in the diagram by numbers within each cell (such as 0 through 12 in partition 0). Partition support for topics provides parallelism.
A key-value pair defined for a single Kafka Streams record. If the record comes directly from a Kafka topic then its key/value are defined as the message key/value.
I see no mention of message keys (org.apache.kafka.clients.producer.ProducerRecord.key) except that they may be used for topic partitioning.
Use of key in Kafka: 1. Partitioning 2. guarantees ordering within a partition 3. Used during log compaction to create offset map
may be more which I haven't learnt yet.. :-)
Am I free to put whatever data I like in the key or are there some special semantics that I should conform to?
Key can be of any type Null, string or any other form with valid serialization mechanism.
do you know if the size of the key matters (e.g. 64 bytes vs 200 bytes)
Size of the key will matter as it will change your overall payload size. Therefore each message utilization on the buffer will change. I can't think of any other impact.
101
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
