What is the best practice for naming kafka topics?

Question

We are new to kafka and we have a few teams working on a few applications that publish/subscribe events to/from each other. Since the kafka topic names are going to be shared across teams, is there any best practice for naming?

Basically we don't want to see team A naming a topic companyname-appname-events while team B naming another topic productname_functionB in totally different styles.

Any suggestions are appreciated !

Note this probably sounds more like the following asked question: What should be naming convention of topic and partition of Kafka? However, the author there was asking something more specific.

Answer 1

https://riccomini.name/how-paint-bike-shed-kafka-topic-naming-conventions helped us answering that same question.

As a summary this article suggest to follow similar best practices to naming databases and tables, and it provides these additional points of advice:

1. Avoid topic names based on things that change
2. Avoid topic names based on information that would be stored in other places
3. Avoid topic names based on their planned consumers/producers. This is essentially a special case of the first advice :D.
4. Decide casing early on, and consider enforcing it or at least check/monitor it. This way you catch offenders early on.

Answer 2

I would like to present an alternative approach to the above answer, which has worked well in practice so far and which does not cause any coupling to the product or application name.

My recommendation:

1. Use of business domains and sub-domains, e.g.:

   public.sales.ecommerce.shoppingcarts

   private.risk.portfolio.analysis.loans.csvimport

2. In addition, domain internal topics can be marked with "private" or external topics (i.e. after quality assurance for example) with "public". Another example:

   private.risk.portfolio.pricingengine.assetpricing

The number of domains and subdomains and whether you need to mark streams as public or private obviously depends on your organization's size. As always, overengineering should be avoided.

I have written a blog post "Topic naming conventions: How do I name my topics? 5 recommendations with examples", which goes into more details.