Authenticating to GKE master in Python

Question

I need to authenticate to a Kubernetes cluster provisioned in GKE using the Kubernetes Python client and the Google Cloud python client. I would prefer not to shell out to gcloud for several reasons:

• relying on the system shell gcloud in a Python script when I have a native Google Cloud library is inelegant
• it requires the system to have gcloud
• I would have to switch users to the relevant ServiceAccount and switch back
• It incurs the cost of starting/joining another process

As such, the workflow of gcloud container clusters get-credentials (which delegates to gcloud config config-helper) will not suffice to get me the API key I need. How do I get the equivalent output with the Google Cloud Python API?

Here is what I have so far:

import kubernetes.client
import googleapiclient.discovery
import base64

# get the cluster object from GKE
gke = googleapiclient.discovery.build('container', 'v1', credentials=config['credentials'])
name = f'projects/{config["project_id"]}/locations/{config["location"]}/{parent}/clusters/{config["name"]}'
gke_clusters = gke.projects().locations().clusters()
gke_cluster = gke_clusters.get(name=name).execute()

# set up Kubernetes Config
kube_config = kubernetes.client.Configuration()
kube_config.host = 'https://{0}/'.format(gke_cluster['endpoint'])
kube_config.verify_ssl = True
#kube_config.api_key['authenticate'] = "don't know what goes here"

# regretably, the Kubernetes client requires `ssl_ca_cert` to be a path, not the literal cert, so I will write it here.
kube_config.ssl_ca_cert = 'ssl_ca_cert'
with open(kube_config.ssl_ca_cert, 'wb') as f:
    f.write(base64.decodestring(gke_cluster['masterAuth']['clusterCaCertificate'].encode()))

# use Kubernetes client to do something
kube_client = kubernetes.client.ApiClient(configuration=kube_config)
kube_v1 = kubernetes.client.CoreV1Api(kube_client)
kube_v1.list_pod_for_all_namespaces(watch=False)

Answer 1

Below is a solution that pulls the access token out of the googleapiclient, rather than copy-pasting things manually.

import googleapiclient.discovery
from tempfile import NamedTemporaryFile
import kubernetes
import base64

def token(*scopes):
    credentials = googleapiclient._auth.default_credentials()
    scopes = [f'https://www.googleapis.com/auth/{s}' for s in scopes]
    scoped = googleapiclient._auth.with_scopes(credentials, scopes)
    googleapiclient._auth.refresh_credentials(scoped)
    return scoped.token

def kubernetes_api(cluster):
    config = kubernetes.client.Configuration()
    config.host = f'https://{cluster["endpoint"]}'

    config.api_key_prefix['authorization'] = 'Bearer'
    config.api_key['authorization'] = token('cloud-platform')

    with NamedTemporaryFile(delete=False) as cert:
        cert.write(base64.decodebytes(cluster['masterAuth']['clusterCaCertificate'].encode()))
        config.ssl_ca_cert = cert.name

    client = kubernetes.client.ApiClient(configuration=config)
    api = kubernetes.client.CoreV1Api(client)

    return api

def run(cluster):
    """You'll need to give whichever account `googleapiclient` is using the 
    'Kubernetes Engine Developer' role so that it can access the Kubernetes API.

    `cluster` should be the dict you get back from `projects.zones.clusters.get`
    and the like"""

    api = kubernetes_api(cluster)
    print(api.list_pod_for_all_namespaces())

Figuring this out took longer than I care to admit. @Ivan's post helped a lot.

Answer 2

In order to authenticate to a GKE cluster, you can use a service account to connect to a project and then a generated secret key from GKE to authenticate to a cluster. Here are the steps:

1. Create a service account in GCP. Go to IAM > Service Accounts > create a service account. Give it a Project Owner role. Once SA is created, create a key and download it as json.
2. Upload key.json to a folder where you have .py script
3. Get API_TOKEN. This is your main question, you can get it by reading a token file:
   • First run kubectl get secrets
   • You will get ‘default-token-xxxxx’
   • run kubectl describe secrets default-token-xxxxx (replace xxxxx with your token name).
   • The token parameter displayed is your “API-KEY”. Copy it inside your script.
4. Creating a script. It is a bit different then yours for few reasons: you need to authenticate to a project first with a service account, then you need to pass the api_token, but also you need to get SSL certificate when authenticating to GKE master.

import base64, pprint
import kubernetes.client
from google.oauth2 import service_account

credentials = service_account.Credentials.from_service_account_file("key.json")
gke = googleapiclient.discovery.build('container', 'v1', credentials=credentials)
name = 'projects/your_project/locations/your_zone/clusters/your_gke_cluster'
gke_clusters = gke.projects().locations().clusters()
gke_cluster = gke_clusters.get(name=name).execute()

kube_config = kubernetes.client.Configuration()
kube_config.host = 'https://{}'.format(gke_cluster['endpoint'])
kube_config.verify_ssl = True

kube_config.api_key['authorization'] = 'your_api_token'
kube_config.api_key_prefix['authorization'] = 'Bearer'

kube_config.ssl_ca_cert = 'ssl_ca_cert'

with open(kube_config.ssl_ca_cert, 'wb') as f:
    f.write(base64.decodestring(gke_cluster['masterAuth']['clusterCaCertificate'].encode()))

kube_client = kubernetes.client.ApiClient(configuration=kube_config)
kube_v1 = kubernetes.client.CoreV1Api(kube_client)
pprint.pprint(kube_v1.list_pod_for_all_namespaces())

Specific fields:

• your_project - from GCP
• your _zone - where gke cluster is created
• your _gke_cluster - GKE cluster name
• your_api_key - what you get in step 3.

This should be enough to authenticate you to a GKE cluster.