Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

what is key schema in schema registry?

I don't have exact idea of key schema, that what it is, and why it must be used as key is auto-generated and we just pass a value(message).

For value, we pass a schema to the AVRO Serialiser and the serialiser gets it's schema id from schema registry and embeds the schema id with the value(message) we have passed(correct me if I am wrong). What happens to key?

Do we also need to pass a key schema? What is the importance of passing a key schema? And, how to pass a key schema?

like image 365
chitwan Avatar asked Jan 29 '23 13:01

chitwan


2 Answers

Kafka messages are key/value pairs. What you set the key is up to you and the requirements of what you are implementing.

The message key is used for partition assignment. Typically you would key a message based on the processing you expect to do, and any strict ordering you want to impose on the data. For example, if you want to have multiple parallel processes in the same consumer group with each process receiving all records for a given customer, you would key on the customer ID.

like image 57
Robin Moffatt Avatar answered Feb 05 '23 19:02

Robin Moffatt


Adding more info with @Robin's answer,

As each kafka message will have a key and value, key can be null or some primitive type value.

If you send a message with string type key and integer type value for topic T, Schema registry creates two subjects: T-key and T-value.

T-key will store the avro schema of the string type key. If there is no key attached to message(null type), it won't register any schema in schema registry.

like image 28
Nishu Tayal Avatar answered Feb 05 '23 17:02

Nishu Tayal