Ingress responding with 'default backend - 404' when using GKE

Question

Using the latest Kubernetes version in GCP (1.6.4), I have the following Ingress definition:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myproject
  namespace: default
  annotations:
    ingress.kubernetes.io/rewrite-target: /
    kubernetes.io/ingress.class: "gce"
spec:
  rules:
  - host: staging.myproject.io
    http:
      paths:
      - path: /poller
        backend:
          serviceName: poller
          servicePort: 8080

Here is my service and deployment:

apiVersion: v1
kind: Service
metadata:
  name: poller
  labels:
    app: poller
    tier: backend
    role: service
spec:
  type: NodePort
  selector:
    app: poller
    tier: backend
    role: service
  ports:
  - port: 8080
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: poller
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: poller
        tier: backend
        role: service
    spec:
      containers:
      - name: poller
        image: gcr.io/myproject-1364/poller:latest
        imagePullPolicy: Always
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: staging
        - name: GET_HOSTS_FROM
          value: dns
        ports:
        - containerPort: 8080

In my /etc/hosts I have a line like:

35.190.37.148 staging.myproject.io

However, I get default backend - 404 when curling any endpoint on staging.myproject.io:

$ curl staging.myproject.io/poller/cache/status
default backend - 404

I have the exact same configuration working locally inside Minikube, with the only difference being the domain (dev.myproject.io), and that works like a charm.

I have read and tried pretty much everything that I could find, including stuff from here and here and here, but maybe I'm just missing something... any ideas?

Answer 1

It does take 5-10 minutes for an Ingress to actually become usable in GKE. In the meanwhile, you can see responses with status codes 404, 502 and 500.

There is an ingress tutorial here: https://cloud.google.com/container-engine/docs/tutorials/http-balancer I recommend following it. Based on what you pasted, I can say the following:

• You use service.Type=NodePort, which is correct.
• I am not sure about the ingress.kubernetes.io/rewrite-target annotation, maybe that's the issue.
• Make sure your application responds 200 OK to GET / request.
• Also I realize you curl http://<ip>/ but your Ingress spec only handles /poller endpoint. So it's normal you get default backend - 404 response while querying /. You didn't configure any backends for / path in your Ingress spec.