increase proxy_send_timeout and proxy_read_timeout ingress nginx

Question

Im running deployments on GKE,

using quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.12.0 image as nginx-ingress-controller

Im trying to increase proxy_send_timeout and proxy_read_timeout following to this link

here is my ingress config:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress
  namespace: production
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "360s"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "360s"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "360s"
    nginx.ingress.kubernetes.io/proxy-body-size: 100m
    nginx.ingress.kubernetes.io/client-body-buffer-size: 100m
spec:
  rules:
  - host: app.my.com
    http:
      paths:
      - backend:
          serviceName: front-app
          servicePort: 80
  - host: api.my.com
    http:
      paths:
      - backend:
          serviceName: backend-app
          servicePort: 80
  - host: api.old.com
    http:
      paths:
      - backend:
          serviceName: backend-app
          servicePort: 80
  tls:
  - hosts:
    - app.my.com
    - api.my.com
    secretName: tls-secret-my-com
  - hosts:
    - api.old.com
    secretName: tls-secret-old-com

still this does not change the proxy_send_timeout and proxy_read_timeout

requests which take longer than 60s (default nginx timeout) are closed

I see this log:

[error] 20967#20967: * upstream prematurely closed connection while reading response header from upstream, client: 123.456.789.12, server: api.my.com, request: "GET /v1/example HTTP/2.0", upstream: "http://11.22.3.44:4000/v3/example", host: "api.my.com", referrer: "https://app.my.com/"

when I go into the nginx pod:

> kubectl exec -it nginx-ingress-controller-xxxx-yyyy -n ingress-nginx -- bash
> cat /etc/nginx/nginx.conf

output:

server {
    server_name _ ;

    listen 80 default_server  backlog=511;
    location / {
        # Custom headers to proxied server

        proxy_connect_timeout                   5s;
        proxy_send_timeout                      60s;
        proxy_read_timeout                      60s;

proxy_send_timeout and proxy_read_timeout are set to 60s and not 360s as I configured on the ingress

so I tried changing manually the timeout on nginx conf, and then I did not get the timeout on the client, but every time the nginx is restarted the values are returned to the default 60s

how can I configure currectly the timeouts on the ingress?

Answer 1

Based on :https://github.com/kubernetes/ingress-nginx/issues/2007

Try to change the value in the annotation to '360'. The value needs to be a number.

Answer 2

As per Kubernetes documentation you should use numerical values treated as a string.

Example:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: webapp-domain-local
  annotations:
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "10"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1"
spec:
  rules:
  - host: webapp.domain.local
    http:
      paths:
      - backend:
          serviceName: webapplication
          servicePort: 8080