How to exempt a directory when using readOnlyRootFilesystem in kubernetes?

Question

I need to block my k8s pods from writing to root folders, with an exemption to /tmp dir. There are 2 reasons I need to write to this dir:

1. Flask needs to write to somewhere. It's trying to write to /tmp and /etc/... and /opt/... , but all of them are blocked because it's under root folder
2. I'm going to need to write to a file for liveness probe, but if the entire file system is blocked, then I can't do it

I'm running kubernetes 1.13.6-gke.13 in GKE

The relevant part from the yaml file:

securityContext:
  runAsUser: 1000
  readOnlyRootFilesystem: true
  runAsNonRoot: true

I expect the pod to be able to write to a predefined folder, maybe a mounted one.

Answer 1

Create a volume mount for /tmp directory.

volumeMounts:
- mountPath: /tmp
  name: tmp

And in Volumes -

volumes:
- emptyDir: {}
  name: tmp

Answer 2

As I understand, you would like to create a POD with access to a local directory. You need to create PV, PVC and POD.

PV definition:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: pv-flaskapp
  labels:
    type: local
spec:
  storageClassName: <your-storageclass-name>
  capacity:
    storage: 3Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/opt/test_flask/app"

PVC definition:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-flaskapp
spec:
  storageClassName: <your-storageclass-name>
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

POD definition:

apiVersion: v1
kind: Pod
metadata:
  name: flaskapp
spec:
  containers:
  - image: flask:latest
    name: flaskapp
    ports:
      - containerPort: 8080
        name: flaskapp
    volumeMounts:
    - mountPath: /usr/local/flask/webapps
      name: test-volume
  volumes:
  - name: test-volume
    persistentVolumeClaim:
      claimName: pvc-flask

Now you can check if everything works fine:

$ kubectl exec -it flaskapp bash
root@flaskapp:/usr/local/flask# mkdir /usr/local/flask/webapps/sample 
root@flaskapp:/usr/local/flask# touch /usr/local/flask/webapps/sample/testfile
root@flaskapp:/usr/local/flask# ls /usr/local/flask/webapps/sample/
testfile

Now when you look at host, you will see the newly created file:

[root@master user]# ls /opt/test_flask/app/sample/
testfile

I hope it will helps you.