k8s/istio - What's the alternative to HTTP caching between services while Envoy doesn't support it?

Question

I'm looking for a fast HTTP response of cacheable data between services in Kubernetes.

Since my apps are already container-native, I don't want to code in my application the logic of caching, neither via cache lib. The apps are focused mainly on business logic.

I've searched how to configure caching between services and didn't find any Istio configuration to do that, only an issue in the Envoy repository Support HTTP caching and these ongoing related PRs: #7198, #9878, also this talk and this design spec.

On the issue page, in the first comment, someone mentions they are using Nginx as a proxy to cache some API calls. But I don't know if it is the right choice.

Is there a simpler alternative to achieve HTTP caching in the mesh?

Answer 1

Recently we developed a solution using istio&golang to cache service to service communications.

The solution called as "reverse proxy sidecar cache".

Basically you deploy another container in your pod (which is cache container). And you configure routing rules to your pod via istio's VirtualServices. The requests you specify in VirtualService goes to your cache container first, it checks cache storage and returns response if data exists otherwise proxies request to application container.

Overall design looks like:

For example here is an istio VirtualService yaml which routes all get requests to cache container (runs on 9191 port):

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: foo
spec:
  gateways:
  - foo-gateway
  hosts:
  - foo
  http:
  - match:
    - method:
        exact: GET
    route:
    - destination:
        host: foo
        port:
          number: 9191
  - route:
    - destination:
        host: foo
        port:
          number: 8080

Here you can find the article we wrote about it: https://medium.com/trendyol-tech/trendyol-platform-team-caching-service-to-service-communications-on-kubernetes-istio-82327589b935

Here is the project repository: https://github.com/Trendyol/sidecache

Answer 2

The Envoy team has merged the related PRs, but it seems it's not fully usable at the time (with redis, e.g).
There is a SimpleHttpCache, but it is not production ready: #9860

Presentation: https://www.youtube.com/watch?v=uIgYxp-SbBw