internal communication Microservices through API Gateway

Question

In a microservice architecture, there is a common pattern called API Gateway.

I know that all communication from outside the API Gateway is used as a single entry point.

But I also would like that the internal communication from microservice to a microservice is going through the API gateway? I mean it's a lot easier to handle than establishing a point to point connection.

So what speaks against to use the API gateway also for the whole internal communication?

Answer 1

I have tried three flavours

1. All communication via API gateway It makes service discovery easy, all communications can be tracked at one point, but it increases latency for services behind the gateway(Not by much but one extra hop). Also you can strip authentication and that means all services even those behind the gateway need to get the auth right (This may not be cons for some apps but for others this may very well be)
2. External services via gateway It helps you strip auth at the gateway. You can force stricter checks on incoming request, you services talk to each other directly (but that means they need to discover the service some how, we use route53 based dns so endpoints for them to hit remains same). Services trust each other and no auth is required for these communications.
3. External/Internal gateways We also had a scenario where we had to get two api gateways, one reason was different kind of checks that were required on two set of gateways and different set of load that each of them had to undergo.

Answer 2

There are two issues with the first approach (API Gateway for both internal and external calls) that you may want to consider:

1. The load on the Gateway service will become higher. If internal services on average make one call to any other internal service, the load on the Gateway service will double. This may result in extra latency not just because of an extra hop but the fact that each request has to go through the extra load on the Gateway service instances. This will force you to augment your Gateway service hardware (horizontal or vertical) for no tangible benefit.

2. Once the load goes higher and touches the peak capacity of the Gateway service instances, these instances may start running out of their resources like worker threads and memory. In general, this situation can be handled by load shedding or throttling of some new requests. This may mean that we may serve only a percentage of requests until the load comes down. However, in our case, not only the new requests are affected but all those in-flight old requests that are waiting for Gateway service resources to get freed up for internal calls are also blocked for-ever till they time out, as these in-flight requests are waiting for original requests (themselves) to complete. Thus we end up with a deadlocked system that will serve no requests at all till the load comes down. If timeouts are not properly implemented, the system may even get permanently deadlocked, till the deadlocked gateway service instances are recycled.

For the second approach where internal services can communicate directly with other internal services, we can use client-side load balancers along with service discovery either through a Discovery Service or a DNS. This will not only perform better but also require lesser hardware than the first approach.