Kubernetes Ingress not adding the application URL for grafana dashboard

Question

I have install the Grafan in my Kubenernetes 1.9 cluster. When I access with my ingress URL (http://sample.com/grafana/ ) getting the first page. After that javascript, css download not adding /grafana to the URL.

here is my ingress rule:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: grafana-ingress-v1
  namespace: monitoring
  annotations:
    ingress.kubernetes.io/rewrite-target: /
    kubernetes.io/ingress.class: nginx
spec:
  tls:
  - hosts:
    - sample.com
    secretName: ngerss-tls
  rules:
  - host: sample.com
    http:
      paths:
      - path: /grafana/
        backend:
          serviceName: grafana-grafana
          servicePort: 80

Here I see the discussion about the same topic. but its not helping my issue.

https://github.com/kubernetes/contrib/issues/860 Below images shows first request goes to /grafana/ but second request didn't get added /grafana/ in the url.

Answer 1

Your ingress rule is correct and nginx creates correct virtual host to forward traffic to grafana's service (I left only needed strings to show):

 server {
    server_name sample.com;
    listen 80;
    listen [::]:80;
    set $proxy_upstream_name "-";

    location ~* ^/grafana/(?<baseuri>.*) {

        set $proxy_upstream_name "default-grafana-grafana-80";

        set $namespace      "default";
        set $ingress_name   "grafana-ingress-v1";


    rewrite /grafana/(.*) /$1 break;
    rewrite /grafana/ / break;
    proxy_pass http://default-grafana-grafana-80;

    }

And yes, when you go to sample.com/grafana/ you get the response from grafana pod, but it redirects to sample.com/login page (you see this from screenshot you provided):

$ curl -v -L http://sample.com/grafana/
*   Trying 192.168.99.100...
* Connected to sample.com (192.168.99.100) port 80 (#0)
> GET /grafana/ HTTP/1.1
> Host: sample.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< Server: nginx/1.13.5
< Date: Tue, 30 Jan 2018 21:55:21 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 29
< Connection: keep-alive
< Location: /login
< Set-Cookie: grafana_sess=c07ab2399d82fef4; Path=/; HttpOnly
< Set-Cookie: redirect_to=%252F; Path=/
< 
* Ignoring the response-body
* Connection #0 to host sample.com left intact
* Issue another request to this URL: 'http://sample.com/login'
* Found bundle for host sample.com: 0x563ff9bf7f20 [can pipeline]
* Re-using existing connection! (#0) with host sample.com
* Connected to sample.com (192.168.99.100) port 80 (#0)
> GET /login HTTP/1.1
> Host: sample.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Server: nginx/1.13.5
< Date: Tue, 30 Jan 2018 21:55:21 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 21
< Connection: keep-alive
< 
* Connection #0 to host sample.com left intact
default backend 404

because by default grafana's root_url is just /:

root_url = %(protocol)s://%(domain)s:%(http_port)s/

and when request redirects to just sample.com nginx forwards it to default backend 404.

Solution:

You need to change root_url grafana's server setting to /grafana/:

root_url = %(protocol)s://%(domain)s:%(http_port)s/grafana/

You can do this changing this setting in grafana's configmap object.