How to configure Ingress request timeouts on GKE

Question

I currently have an Ingress configured on GKE (k8s 1.2) to forward requests towards my application's pods. I have a request which can take a long time (30 seconds) and timeout from my application (504). I observe that when doing so the response that i receive is not my own 504 but a 502 from what looks like the Google Loadbalancer after 60 seconds.

I have played around with different status codes and durations, exactly after 30 seconds i start receiving this weird behaviour regardless of statuscode emitted.

Anybody have a clue how i can fix this? Is there a way to reconfigure this behaviour?

Answer 1

Beginning with 1.11.3-gke.18, it is possible to configure timeout settings in kubernetes directly.

First add a backendConfig:

apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
  name: my-bsc-backendconfig
spec:
  timeoutSec: 40

Then add an annotation in Service to use this backendConfig:

apiVersion: v1
kind: Service
metadata:
  name: my-bsc-service
  labels:
    purpose: bsc-config-demo
  annotations:
    beta.cloud.google.com/backend-config: '{"ports": {"80":"my-bsc-backendconfig"}}'
spec:
  type: NodePort
  selector:
    purpose: bsc-config-demo
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080

And viola, your ingress load balancer now has a timeout of 40 second instead of the default 30 seconds.

See https://cloud.google.com/kubernetes-engine/docs/how-to/configure-backend-service#creating_a_backendconfig

Answer 2

When creating an ingress on GKE the default setup is that a GLBC HTTP load balancer will be created with the backends that you supplied. Default it is configured at a 30 second timeout for your application to handle the request.

If you need a longer timeout you have to edit this manually after setup in the backends of your HTTP Load balancer in the google cloud console.