Fail to connect to kubectl from client-go - /serviceaccount/token: no such file

Question

I am using golang lib client-go to connect to a running local kubrenets. To start with I took code from the example: out-of-cluster-client-configuration.

Running a code like this: $ KUBERNETES_SERVICE_HOST=localhost KUBERNETES_SERVICE_PORT=6443 go run ./main.go results in following error:

panic: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory

goroutine 1 [running]:

/var/run/secrets/kubernetes.io/serviceaccount/

I am not quite sure which part of configuration I am missing. I've researched following links :

• https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins

• https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/

But with no luck. I guess I need to either let the client-go know which token/serviceAccount to use, or configure kubectl in a way that everyone can connect to its api.

Here's status of my kubectl though some commands results:

$ kubectl config view

apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true
    server: https://localhost:6443
  name: docker-for-desktop-cluster
contexts:
- context:
    cluster: docker-for-desktop-cluster
    user: docker-for-desktop
  name: docker-for-desktop
current-context: docker-for-desktop
kind: Config
preferences: {}
users:
- name: docker-for-desktop
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

$ kubectl get serviceAccounts

NAME        SECRETS   AGE
default     1         3d
test-user   1         1d

$ kubectl describe serviceaccount test-user

Name:                test-user
Namespace:           default
Labels:              <none>
Annotations:         <none>
Image pull secrets:  <none>
Mountable secrets:   test-user-token-hxcsk
Tokens:              test-user-token-hxcsk
Events:              <none>

$ kubectl get secret test-user-token-hxcsk -o yaml

apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0......=
  namespace: ZGVmYXVsdA==
  token: ZXlKaGJHY2lPaUpTVXpJMU5pSX......=
kind: Secret
metadata:
  annotations:
    kubernetes.io/service-account.name: test-user
    kubernetes.io/service-account.uid: 984b359a-6bd3-11e8-8600-XXXXXXX
  creationTimestamp: 2018-06-09T10:55:17Z
  name: test-user-token-hxcsk
  namespace: default
  resourceVersion: "110618"
  selfLink: /api/v1/namespaces/default/secrets/test-user-token-hxcsk
  uid: 98550de5-6bd3-11e8-8600-XXXXXX
type: kubernetes.io/service-account-token

Answer 1

This answer could be a little outdated but I will try to give more perspective/baseline for future readers that encounter the same/similar problem.

TL;DR

The following error:

panic: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory

is most likely connected with the lack of token in the /var/run/secrets/kubernetes.io/serviceaccount location when using in-cluster-client-configuration. Also, it could be related to the fact of using in-cluster-client-configuration code outside of the cluster (for example running this code directly on a laptop or in pure Docker container).

You can check following commands to troubleshoot your issue further (assuming this code is running inside a Pod):

• $ kubectl get serviceaccount X -o yaml:
  • look for: automountServiceAccountToken: false
• $ kubectl describe pod XYZ
  • look for: containers.mounts and volumeMounts where Secret is mounted

Citing the official documentation:

Authenticating inside the cluster

This example shows you how to configure a client with client-go to authenticate to the Kubernetes API from an application running inside the Kubernetes cluster.

client-go uses the Service Account token mounted inside the Pod at the /var/run/secrets/kubernetes.io/serviceaccount path when the rest.InClusterConfig() is used.

-- Github.com: Kubernetes: client-go: Examples: in cluster client configuration

If you are authenticating to the Kubernetes API with ~/.kube/config you should be using the out-of-cluster-client-configuration.

---

Additional information:

I've added additional information for more reference on further troubleshooting when the code is run inside of a Pod.

• automountServiceAccountToken: false

In version 1.6+, you can opt out of automounting API credentials for a service account by setting automountServiceAccountToken: false on the service account:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: go-serviceaccount
automountServiceAccountToken: false

In version 1.6+, you can also opt out of automounting API credentials for a particular pod:

apiVersion: v1
kind: Pod
metadata:
  name: sdk
spec:
 serviceAccountName: go-serviceaccount
 automountServiceAccountToken: false

-- Kubernetes.io: Docs: Tasks: Configure pod container: Configure service account

• $ kubectl describe pod XYZ:

When the servicAccount token is mounted, the Pod definition should look like this:

<-- OMITTED --> 
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from go-serviceaccount-token-4rst8 (ro)
<-- OMITTED --> 
Volumes:
  go-serviceaccount-token-4rst8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  go-serviceaccount-token-4rst8
    Optional:    false

If it's not:

<-- OMITTED --> 
    Mounts:         <none>
<-- OMITTED --> 
Volumes:            <none>

---

Additional resources:

• Kubernetes.io: Docs: Reference: Access authn authz: Authentication