Are you trying to mount a directory onto a file (or vice-versa) with kuberneters/configMap?

Question

I followed this post Kubernetes configMap - only one file to pass a config file to a deployment, but got an error. Why?

The config file config-prom-prometheus.yml:

scrape_configs:
- job_name: job-leo-prometheus
  kubernetes_sd_configs:
  - role: endpoints

The .yaml file prom-prometheus.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prom-prometheus-deployment
spec:
  selector:
    matchLabels:
      app: prom-prometheus
  replicas: 1
  template:
    metadata:
      labels:
        app: prom-prometheus
    spec:
      containers:
      - name: prom-prometheus
        image: 127.0.0.1:30400/prom/prometheus
        ports:
        - name: port9090
          containerPort: 9090
        volumeMounts:
        - name: volume-prometheus
          mountPath: /etc/prometheus/prometheus.yml
          subPath: prometheus.yml
      volumes:
      - name: volume-prometheus
        configMap:
          name: config-prom

---
apiVersion: v1
kind: Service
metadata:
  name: prom-prometheus
spec:
  type: NodePort
  ports:
  - name: port9090
    protocol: TCP
    port: 9090
    targetPort: 9090
    nodePort: 30090
  selector:
    app: prom-prometheus

Commands:

kubectl create configmap config-prom --from-file=config-prom-prometheus.yml
kubectl -f prom-prometheus.yaml apply

Results:

Events:
  Type     Reason                 Age               From               Message
  ----     ------                 ----              ----               -------
  Normal   Scheduled              17s               default-scheduler  Successfully assigned prom-prometheus-deployment-66887dcdbf-bfqd4 to minikube
  Normal   SuccessfulMountVolume  17s               kubelet, minikube  MountVolume.SetUp succeeded for volume "default-token-ml6w5"
  Normal   SuccessfulMountVolume  17s               kubelet, minikube  MountVolume.SetUp succeeded for volume "volume-prometheus"
  Warning  Failed                 9s                kubelet, minikube  Error: failed to start container "prom-prometheus": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/var/lib/kubelet/pods/ec99da92-e994-11e8-a578-08002742f2a3/volume-subpaths/volume-prometheus/prom-prometheus/0\\\" to rootfs \\\"/var/lib/docker/overlay2/12c7da1c07c55fe2ec5dff61e5c457fa8aeaa32d47232c28a1d7e127c4f81bf0/merged\\\" at \\\"/var/lib/docker/overlay2/12c7da1c07c55fe2ec5dff61e5c457fa8aeaa32d47232c28a1d7e127c4f81bf0/merged/etc/prometheus/prometheus.yml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Normal   Pulling                7s (x2 over 13s)  kubelet, minikube  pulling image "127.0.0.1:30400/prom/prometheus"
  Normal   Pulled                 7s (x2 over 13s)  kubelet, minikube  Successfully pulled image "127.0.0.1:30400/prom/prometheus"
  Normal   Created                6s (x2 over 10s)  kubelet, minikube  Created container
  Warning  Failed                 4s                kubelet, minikube  Error: failed to start container "prom-prometheus": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/var/lib/kubelet/pods/ec99da92-e994-11e8-a578-08002742f2a3/volume-subpaths/volume-prometheus/prom-prometheus/0\\\" to rootfs \\\"/var/lib/docker/overlay2/7b07728ae4439e4d479386eab6b042948e2cb586c54171941f24d03352a7c8b4/merged\\\" at \\\"/var/lib/docker/overlay2/7b07728ae4439e4d479386eab6b042948e2cb586c54171941f24d03352a7c8b4/merged/etc/prometheus/prometheus.yml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

Answer 1

This is not well documented but as per my experience name of configmap yaml (config-prom-prometheus.yml in your case) should be the same as mountPath and subPath in Deployment.

If you use subPath: prometheus.yml - rename config-prom-prometheus.yml to prometheus.yml and try again.

Answer 2

This is probably slightly rewording the accepted answer, but one thing that confused me was that the mount path expected full path of to the target file.

subPath is the key in the volume (e.g. config map) for the source.

mountPath is target full path for the file.

For some reason my brain was thinking mountPath was the target directory and the subPath would infer the target file name. All I had to do was add the filename at the end of the mountPath.

For example:

Bad:

- name: config-volume
  mountPath: "/opt/config/path/"
  subPath: "config-file.json"

Good:

- name: config-volume
  mountPath: "/opt/config/path/config-file.json"
  subPath: "config-file.json"