414 Request URI too long using Nginx Ingress on Kubernetes

Question

We are using kubernetes/ingress-nginx for our Azure AKS instance. I have a URI that is 9kb long approximately (it contains a post_logout_redirect_uri and a very long id_token_hint for our Identity server, running in .Net core 2.2).

However, I cannot get past the ingress as nginx is rejecting the query with 414 URI Too Long. I can see the request in the Nginx logs but not on the Identity server logs, so it is clearly getting bounced before.

I have tried to update the nginx configuration using config map, but without success. The settings are applied (and have helped me fix other issues before). However, in this case nothing I try seems to have worked. Here is the config map I'm using:

apiVersion: v1
data:
  http2-max-header-size: "64k"
  http2-max-field-size: "32k"
  proxy-body-size: "100m"
  client-header-buffer-size: "64k"
  large-client-header-buffers: "4 64k"
kind: ConfigMap
metadata:
  name: nginx-ingress-controller
  namespace: kube-system

Here are the ingress annotations for the Identity server:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress-name
  annotations:
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt
    nginx.ingress.kubernetes.io/send_timeout: "180"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "180"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "180"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "180"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-headers: "authorization,content-type"
    nginx.ingress.kubernetes.io/proxy-body-size: 250m
    nginx.ingress.kubernetes.io/proxy-buffer-size: "64k"

Finally, if I check the nginx config on the pod it does contain my updated values, in the global config section.

...
 keepalive_timeout  75s;
 keepalive_requests 100;

 client_body_temp_path           /tmp/client-body;
 fastcgi_temp_path               /tmp/fastcgi-temp;
 proxy_temp_path                 /tmp/proxy-temp;
 ajp_temp_path                   /tmp/ajp-temp;

 client_header_buffer_size       64k;
 client_header_timeout           60s;
 large_client_header_buffers     4 64k;
 client_body_buffer_size         8k;
 client_body_timeout             60s;

 http2_max_field_size            32k;
 http2_max_header_size           64k;
 http2_max_requests              1000;

 types_hash_max_size             2048;
 server_names_hash_max_size      1024;
 server_names_hash_bucket_size   64;
 map_hash_bucket_size            64;

 proxy_headers_hash_max_size     512;
 proxy_headers_hash_bucket_size  64;

 variables_hash_bucket_size      128;
 variables_hash_max_size         2048;

 underscores_in_headers          off;
 ignore_invalid_headers          on;
...

Any info or suggestions would be appreciated, thanks!

Answer 1

I also tried the following annotations:

nginx.ingress.kubernetes.io/large_client_header_buffers: 200m
nginx.ingress.kubernetes.io/proxy-body-size: 200m

They didn't help, what did help is the snippet I added in the Ingress controller yaml:

nginx.ingress.kubernetes.io/server-snippet: |
  http2_max_header_size 256k;
  http2_max_field_size 256k;