How to run kubectl within a job in a namespace?

Question

Hi I saw this documentation where kubectl can run inside a pod in the default pod. Is it possible to run kubectl inside a Job resource in a specified namespace? Did not see any documentation or examples for the same..

When I tried adding serviceAccounts to the container i got the error:

Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:my-namespace:internal-kubectl" cannot list resource "pods" in API group "" in the namespace "my-namespace"

This was when i tried sshing into the container and running the kubctl.

Editing the question.....

As I mentioned earlier, based on the documentation I had added the service Accounts, Below is the yaml:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: internal-kubectl  
  namespace: my-namespace   
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: modify-pods
  namespace: my-namespace
rules:
  - apiGroups: [""]
    resources:
      - pods
    verbs:
      - get
      - list
      - delete      
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: modify-pods-to-sa
  namespace: my-namespace
subjects:
  - kind: ServiceAccount
    name: internal-kubectl
roleRef:
  kind: Role
  name: modify-pods
  apiGroup: rbac.authorization.k8s.io      
---
apiVersion: batch/v1
kind: Job
metadata:
  name: testing-stuff
  namespace: my-namespace
spec:
  template:
    metadata:
      name: testing-stuff
    spec:
      serviceAccountName: internal-kubectl
      containers:
      - name: tester
        image: bitnami/kubectl
        command:
         - "bin/bash"
         - "-c"
         - "kubectl get pods"
      restartPolicy: Never 

On running the job, I get the error:

Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:my-namespace:internal-kubectl" cannot list resource "pods" in API group "" in the namespace "my-namespace"

Answer 1

Is it possible to run kubectl inside a Job resource in a specified namespace? Did not see any documentation or examples for the same..

A Job creates one or more Pods and ensures that a specified number of them successfully terminate. It means the permission aspect is the same as in a normal pod, meaning that yes, it is possible to run kubectl inside a job resource.

TL;DR:

• Your yaml file is correct, maybe there were something else in your cluster, I recommend deleting and recreating these resources and try again.
• Also check the version of your Kubernetes installation and job image's kubectl version, if they are more than 1 minor-version apart, you may have unexpected incompatibilities

---

Security Considerations:

• Your job role's scope is the best practice according to documentation (specific role, to specific user on specific namespace).
• If you use a ClusterRoleBinding with the cluster-admin role it will work, but it's over permissioned, and not recommended since it's giving full admin control over the entire cluster.

---

Test Environment:

• I deployed your config on a kubernetes 1.17.3 and run the job with bitnami/kubectl and bitnami/kubectl:1:17.3. It worked on both cases.
• In order to avoid incompatibility, use the kubectl with matching version with your server.

---

Reproduction:

$ cat job-kubectl.yaml 
apiVersion: batch/v1
kind: Job
metadata:
  name: testing-stuff
  namespace: my-namespace
spec:
  template:
    metadata:
      name: testing-stuff
    spec:
      serviceAccountName: internal-kubectl
      containers:
      - name: tester
        image: bitnami/kubectl:1.17.3
        command:
         - "bin/bash"
         - "-c"
         - "kubectl get pods -n my-namespace"
      restartPolicy: Never 

$ cat job-svc-account.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: internal-kubectl  
  namespace: my-namespace   
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: modify-pods
  namespace: my-namespace
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "delete"]      
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: modify-pods-to-sa
  namespace: my-namespace
subjects:
  - kind: ServiceAccount
    name: internal-kubectl
roleRef:
  kind: Role
  name: modify-pods
  apiGroup: rbac.authorization.k8s.io

• I created two pods just to add output to the log of get pods.

$ kubectl run curl --image=radial/busyboxplus:curl -i --tty --namespace my-namespace
the pod is running
$ kubectl run ubuntu --generator=run-pod/v1 --image=ubuntu -n my-namespace
pod/ubuntu created

• Then I apply the job, ServiceAccount, Role and RoleBinding

$ kubectl get pods -n my-namespace
NAME                    READY   STATUS      RESTARTS   AGE
curl-69c656fd45-l5x2s   1/1     Running     1          88s
testing-stuff-ddpvf     0/1     Completed   0          13s
ubuntu                  0/1     Completed   3          63s

• Now let's check the testing-stuff pod log to see if it logged the command output:

$ kubectl logs testing-stuff-ddpvf -n my-namespace
NAME                    READY   STATUS    RESTARTS   AGE
curl-69c656fd45-l5x2s   1/1     Running   1          76s
testing-stuff-ddpvf     1/1     Running   0          1s
ubuntu                  1/1     Running   3          51s

As you can see, it has succeeded running the job with the custom ServiceAccount.

Let me know if you have further questions about this case.