I have a kubernetes cluster, where I have deployed an nginx ingress controller using the helm nginx-ingress chart.
I need to add some custom config to the nginx.conf file that is generated in the nginx-controller-pod, and I am seeing an issue where if I add a one line option such as proxy-buffer-size: "512k"
I can see this reflected in the nginx.conf file and everything works as expected.
However, if I attempt to add a snippet to accomplish the same thing:
location-snippet: |
proxy_buffer_size "512k";
It is as though this is ignored by the nginx.conf file and the proxy_buffer_size
setting remains at it's default value.
I need to be able to add http-snippet
, server-snippet
and location-snippet
overrides but whether I try to add them to the ConfigMap or as an annotation in the Ingress.yaml file they are always ignored.
My Ingress yaml file:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/secure-backends: "true"
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/location-snippet: |
proxy_buffer_size 512k; --This does not update the nginx.conf
spec:
tls:
- hosts:
- my.app.co.uk
secretName: tls-secret
rules:
- host: my.app.co.uk
http:
paths:
- path: /
backend:
serviceName: myappweb-service
servicePort: 80
My nginx config map:
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app: nginx-ingress
chart: nginx-ingress-0.28.3
component: controller
heritage: Tiller
release: nginx-ingress
name: nginx-ingress-controller
namespace: default
data:
proxy-buffer-size: "512k" -- this works and updates the nginx.conf
location-snippet: |
proxy_buffers 4 512k; -- this does not update the nginx.conf
server-snippet: | -- this does not update the nginx.conf
location /messagehub {
proxy_set_header Upgrade $http_upgrade;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_set_header Connection "upgrade";
proxy_cache_bypass $http_upgrade;
}
Snippets allow you to insert raw NGINX config into different contexts of the NGINX configurations that the Ingress Controller generates. These should be used as a last-resort solution in cases where annotations and ConfigMap entries cannot help.
The configuration-snippet is to add configs to locations. If you want to add a custom location to the server context, you should use the server-snippet instead: Using the annotation nginx.ingress.kubernetes.io/server-snippet it is possible to add custom configuration in the server configuration block.
proxy-connect-timeout : this defines the timeout for establishing a connection with a proxied server. The default value is 60 seconds, and the timeout typically cannot exceed 75 seconds. Check here for more information. proxy-send-timeout : this will set a timeout for transmitting a request to the proxied server.
This size can be configured by the parameter client_max_body_size and is set to 1m (1 Megabyte) by default. To configure this setting globally for all Ingress rules, the proxy-body-size value may be set in the NGINX ConfigMap.
It turns out that my problem was due to the content of the snippet that I was applying. Every time you run kubectl apply -f myconfigmap.yaml
, a validation is run against the changes that you are trying to apply to the nginx.conf. When this validation fails, it fails silently and there is nothing to alert you to this in the terminal.
In fact, you still get the configmap/nginx-ingress-controller configured
message.
For example, when I add this to the config map, it updates the nginx.conf as expected:
http-snippet: |
sendfile on;
However, when I add this, nothing changes:
http-snippet: |
sendfile on;
tcp_nopush on;
The reason being that this has failed validation, but the only way to find that out is to look at the logs of the nginx ingress controller pod. In this instance I see:
Error: exit status 1
2018/10/16 07:45:49 [emerg] 470#470: "tcp_nopush" directive is duplicate in
/tmp/nginx-cfg468835321:245
nginx: [emerg] "tcp_nopush" directive is duplicate in /tmp/nginx-cfg468835321:245
nginx: configuration file /tmp/nginx-cfg468835321 test failed
So I was duplicating a directive that already existed.
If you'd like to modify your Kubernetes Ingress
the annotation options are these:
nginx.ingress.kubernetes.io/configuration-snippet
for an nginx location block snippetnginx.ingress.kubernetes.io/server-snippet
for a snippet in the nginx config service blockLooks like you are using nginx.org/location-snippets:
for that case.
There's also a YAML invalid syntax on nginx config example and also you should use plurals as in server-snippets
according to this example. There's a typo in the docs as of this writing. Opened this ticket to follow up.
It should be something like this:
server-snippets: |
location /messagehub {
proxy_set_header Upgrade $http_upgrade;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_set_header Connection "upgrade";
proxy_cache_bypass $http_upgrade;
}
instead of this:
server-snippet: |
location /messagehub {
proxy_set_header Upgrade $http_upgrade;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_set_header Connection "upgrade";
proxy_cache_bypass $http_upgrade;
}
Notice the indentation of the last curly brace.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With