Automatic subdirectories in Kubernetes configmaps?

Question

(A very similar question was asked about 2 years ago, though it was specifically about secrets, I doubt the story is any different for configmaps... but at the least, I can present the use case and why the existing workarounds aren't viable for us.)

Given a simple, cut-down deployment.yaml:

apiVersion: apps/v1beta1
kind: Deployment
metadata: 
  name: example
spec:
  template: 
    spec:
      containers:
      - name: example
        volumeMounts:
        - name: vol
          mountPath: /app/Configuration
      volumes:
        - name: vol
          configMap:
            name: configs

and the matching configmap.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: configs
  labels:
    k8s-app: example
data:
    example1.json: |-
        {
            "key1": "value1"
        }

    example2.json: |-
        {
            "key2": "value2"
        }

the keys in configmap.yaml, whatever they may be, are simply created as files, without deployment.yaml needing to be modified or have any specifics other than the mountPath.

The problem is that the actual structure has subfolders to handle region-specific values that override the root ones:

Configuration \ example1.json
Configuration \ example2.json
Configuration \ us \ example1.json
Configuration \ us \ ca \ example2.json

The number and nature of these could obviously vary, for as many different countries and regions imaginable and for each separately configured module. The intent was to provide a tool to the end user that would allow them to set up and manage these configurations, which would behind the scenes automatically generate the configmap.yaml and update it in kubernetes.

However, unless there's a trick I haven't found yet, this seems to be outside of kubernetes's abilities, in a couple ways.

First of all, there is no syntax that allows one to specify configmap keys that are directories, nor include a subdirectory path in a key:

data:
    # one possible approach (currently complains that it doesn't validate '[-._a-zA-Z0-9]+')
    /us/example1.json: |-
        {
            "key1": "value1"
        }

    # another idea; this obviously results in 'invalid type for io.k8s.api.core.v1.ConfigMap.data: got "map", expected "string"'
    us:
        example2.json: |-
            {
                "key2": "value2"
            }

So what are our options to accomplish this?

Wellll, we could map the keys to specific locations using the items: -key: path: approach in the deployment.yaml's volumes: -configMap: node,

and/or generate several nodes in the deployment.yaml's volumeMounts: node,

using either subPath: (which is basically the same as using items: -key: -path: in the volumes: configMap:),

or individual separate configmaps for each subdirectory, and mounting them all as different volumes in the deployment.yaml.

All of these methods would require massive and incredibly verbose changes to the deployment.yaml, leaking out knowledge it shouldn't have any reason to know about, making it mutable and continually re-generated rather than static, complicating rolling out settings updates to deployed pods, etc. etc. etc. It's just Not Good. And all of that just to have mapped one directory, just because it contains subdirectories...

Surely this CAN'T be the way it's SUPPOSED to work? What am I missing? How should I proceed?

Answer 1

From a "container-native" perspective, having a large file system tree of configuration files that the application processes at startup to arrive at its canonical configuration is an anti-pattern. Better to have a workflow that produces a single file, which can be stored in a ConfigMap and easily inspected in its final form. See, for instance, nginx ingress.

But obviously not everyone is rewriting their apps to better align with the kubernetes approach. The simplest way then to get a full directory tree of configuration files into a container at deploy time is to use initContainers and emptyDir mounts.

Package the config file tree into a container (sometimes called a "data-only" container), and have the container start script just copy the config tree into the emptyDir mount. The application can then consume the tree as it expects to.