Microservices : aggregate data : is there some good patterns?

Question

I have the following use case for a micro-services architecture.

My issue is that in the current situation I have 3 microservices, and a APIGateway.

At the end, the gateway has to do a lot of queries before aggregation (composing) the data from the 3 services. Because the 3 microservices are only providing basic set of datas.

Please, check the picture for more details !

Is it a good pattern ? Is there other pattern ?

Answer 1

The above is a common problem with microservices - separation of domains. Though each service performs tasks in a distinct domain, they contain relationships to data "owned" by other services. The way you're currently solving it is actually one of the easier ways to manage the implementation since it's not terribly complex, but there are solutions that are more efficient.

1. Replication

One of the things that is tough to wrap your head around when adopting microservices is that data replication is NOT a bad thing. Though it can feel like data is dangling in multiple places, the creator of a service is actually afforded more autonomy by replicating the data they need into their own managed database.

If you were to publish events into a shared queue, you could setup read/sync processes for microservices that depend on data from other services to read said events and mirror the data into private databases. In your example, this would mean that the Catalog service would be able to return the fully populated models your API gateway returns w/out the need to call other services.

Source: The Hardest Part About Microservices: Your Data

2. Turning the database inside-out

Another increasingly common strategy is even harder to wrap your head around, but provides a lot of long-term value for evolving microservices architectures. Instead of thinking of the databases for your services as persistent storage, think of them as views. All writes can go to a central source, and services instead define how to map that central data into a persistent view for their service for reads. This let's services define views that include data written by other services - again, without restricting autonomy for service creators.

Source: Turning the database inside-out with Apache Samza

3. Merge the services

Depending on your expectation of growth, it might not be worth implementing either of the above solutions. Given the relational nature of the data, you may simply want to merge the services together into a service that can use joins for querying and build the models themselves.

Note: all of the above options share one major similarity - don't depend on a custom gateway to intimately understand each service and build out model relationships. Make sure each service has enough information and context to perform a meaningful task on its own.