Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

One topic per event producer VS One topic shared across multiple producers messaging architecture

This is a bit of general question because it would apply not only to my scenario (with Azure Service Bus) but to any event bus in the context of publish/subscribers with events.

The question is: Is there any preference towards having an architecture/topology where topics must not be shared between producers? In other words: one topic per event producer VS one topic shared across multiple producers?

I have one clear preference: One topic should be owned and accessed only by one producer and if other producers. But I seem to be the only one in a team with this opinion, while others don't seem to have any problem in sharing the same topic between different event producers "for simplicity", and I cannot really argue in terms of technical viability..

I am hoping to find valuable answers and good practices maybe from a more technical point of view, because my reasoning is from a more business/organization point of view as I come from a DDD background and the others don't..

  • A topic is an output box for a one-to-many communication (which I interpret as one event publisher, multiple subscribers)
  • A topic could deal with different types of event messages as long as they're somehow related (and this is very relative, of course)
  • In DDD, there is a concept of bounded context and I like seeing microservices/modules as a way of implementing these bounded contexts. Therefore, even if some other services "think" they want to publish something related and want to access a shared topic to publish, I'd argue that they belong in a different bounded context and they should have their own topic.
  • If multiple producers really belong in the same bounded contexts, then I'd argue that only one service (or infra module) should be in charge of publishing the events that have happened in the bounded context.
  • It's possible that a producer wants to also consume events from other producers (it's also a subscriptor). It wouldn't make sense that a producer subscribes to the same topic and has to discriminate messages depending on whether they were produced by itself or others.

As you can see, from a DDD-ish point of view, multiple producers with the need of publishing in the same topic would trigger a design-smell. I'm not saying it cannot be done, I'm trying to find out whether it should be avoided ALSO from a technical point of view.

Anyone with some hands-on experience on this?

PS: There is a similar question about Kafka but I don't think it's exactly the same as Kafka uses a different technical approach for the publisher-subscriber


UPDATE 1: I don't know NServiceBus, but I've worked a bit with MassTransit and when leveraging the topology creation to MassTransit (which is the only way afaik), it creates different topics not only per producer but per message type.

like image 362
diegosasw Avatar asked Mar 11 '20 09:03

diegosasw


Video Answer


1 Answers

Just to add a few more reasons why I think we should always use one topic per producer as much as we can:

  • if multiple producers publish the same event to the same topic, there is no clear ownership and authority of the event. This is similar to your point on bounded context.
  • when multiple producers publish the same event, it creates a coupling of event schema among those producers. Now that event schema cannot be evolved without affecting all producers at once. Those producers can use event schema versioning but there's still contention when multiple producers updating the same event schema (and produce different versions of event schema)
like image 162
Phuong Nguyen Avatar answered Oct 23 '22 00:10

Phuong Nguyen