How to enable startup probe on GKE 1.16?

Question

I created a deployment with liveness and readiness probes and initial delay which works fine. If I want to replace the initial delay with a startup probe the startupProbe key and its nested elements are never included in the deployment descrioptor when created with kubectl apply and get deleted from the deployment yaml in the GKE deployment editor after saving.

An example:

apiVersion: v1
kind: Namespace
metadata:
  name: "test"
---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres-sleep
  namespace: test
spec:
  selector:
    matchLabels:
      app: postgres-sleep
  replicas: 2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 50%
  template:
    metadata:
      labels:
        app: postgres-sleep
    spec:
      containers:
        - name: postgres-sleep
          image: krichter/microk8s-startup-probe-ignored:latest
          ports:
            - name: postgres
              containerPort: 5432
          readinessProbe:
            tcpSocket:
              port: 5432
            periodSeconds: 3
          livenessProbe:
            tcpSocket:
              port: 5432
            periodSeconds: 3
          startupProbe:
            tcpSocket:
              port: 5432
            failureThreshold: 60
            periodSeconds: 10
---

apiVersion: v1
kind: Service
metadata:
  name: postgres-sleep
  namespace: test
spec:
  selector:
    app: httpd
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
---

with krichter/microk8s-startup-probe-ignored:latest being

FROM postgres:11
CMD sleep 30 && postgres

I'm reusing this example from the same issue with microk8s where I could solve it by changing the kubelet and kubeapi-server configuration files (see https://github.com/ubuntu/microk8s/issues/770 in case you're interested). I assume this is not possible with GKE clusters as they don't expose these files, probably for good reasons.

I assume that the feature needs to be enable since it's behind a feature gate. How can I enable it on Google Kubernetes Engine (GKE) clusters with version >= 1.16? Currently I'm using the default from the regular channel 1.16.8-gke.15.

Answer 1

As I mentioned in my comments, I was able to reproduce the same behavior in my test environment, and after some researches I found the reason.

In GKE, features gates are only permitted if you are using an Alpha Cluster. You can see a complete list of feature gates here

I've created an alpha cluster and applied the same yaml, it works for me, the startupProbe is there in the place.

So, you will only be able to use startupProbe in a GKE Alpha clusters, follow this documentation to create a new one.

Be aware of the limitations in alpha clusters:

• Alpha clusters have the following limitations:
• Not covered by the GKE SLA
• Cannot be upgraded
• Node auto-upgrade and auto-repair are disabled on alpha clusters
• Automatically deleted after 30 days
• Do not receive security updates

Also, Google don't recommend use for production workloads:

Warning: Do not use Alpha clusters or alpha features for production workloads. Alpha clusters expire after thirty days and do not receive security updates. You must migrate your data from alpha clusters before they expire. GKE does not automatically save data stored on alpha clusters.