Populate ConfigMap by importing data from file in k8s

Question

I have a requirement where i push bunch of key value pairs to a text/json file. Post that, i want to import the key value data into a configMap and consume this configMap within a POD using kubernetes-client API's.

Any pointers on how to get this done would be great.

TIA

Answer 1

You can do it in two ways.

Create ConfigMap from file as is.

In this case you will get ConfigMap with filename as a key and filedata as a value.

For example, you have file your-file.json with content {key1: value1, key2: value2, keyN: valueN}.

And your-file.txt with content

key1: value1 key2: value2 keyN: valueN

kubectl create configmap name-of-your-configmap --from-file=your-file.json
kubectl create configmap name-of-your-configmap-2 --from-file=your-file.txt

As result:

apiVersion: v1
kind: ConfigMap
metadata:
  name: name-of-your-configmap
data:
  your-file.json: |
    {key1: value1, key2: value2, keyN: valueN}

apiVersion: v1
kind: ConfigMap
metadata:
  name: name-of-your-configmap-2
data:
  your-file.txt: |
    key1: value1
    key2: value2
    keyN: valueN

After this you can mount any of ConfigMaps to a Pod, for example let's mount your-file.json:

apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: k8s.gcr.io/busybox
      command: [ "/bin/sh","-c","cat /etc/config/keys" ]
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: name-of-your-configmap
        items:
        - key: your-file.json
          path: keys
restartPolicy: Never

Now you can get any information from your /etc/config/your-file.json inside the Pod. Remember that data is read-only.

Create ConfigMap from file with environment variables.

You can use special syntax to define pairs of key: value in file. These syntax rules apply:

• Each line in a file has to be in VAR=VAL format.
• Lines beginning with # (i.e. comments) are ignored.
• Blank lines are ignored.
• There is no special handling of quotation marks (i.e. they will be part of the ConfigMap value)).

You have file your-env-file.txt with content

key1=value1 key2=value2 keyN=valueN

kubectl create configmap name-of-your-configmap-3 --from-env-file=you-env-file.txt

As result:

apiVersion: v1
kind: ConfigMap
metadata:
  name: name-of-your-configmap-3
data:
  key1: value1
  key2: value2
  keyN: valueN

Now you can use ConfigMap data as Pod environment variables:

apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod-2
spec:
  containers:
    - name: test-container
      image: k8s.gcr.io/busybox
      command: [ "/bin/sh", "-c", "env" ]
      env:
        - name: SPECIAL_LEVEL_KEY
          valueFrom:
            configMapKeyRef:
              name: name-of-your-configmap-3
              key: key1
        - name: LOG_LEVEL
          valueFrom:
            configMapKeyRef:
              name: name-of-your-configmap-3
              key: key2
        - name: SOME_VAR
          valueFrom:
            configMapKeyRef:
              name: name-of-your-configmap-3
              key: keyN
  restartPolicy: Never

Now you can use these variables inside the Pod.

For more information check for documentation

Answer 2

I can also recommend Kustomize for this task. You can use it as part of your deployment pipeline to generate the K8s configuration (not only ConfigMaps, but also Deployments, NetworkPolicies, Services etc.).

In kustomize you'd need a ConfigMapGenerator. There are different options. In your case env is suitable.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
# generate a ConfigMap named my-system-env-<some-hash> where each key/value pair in the
# env.txt appears as a data entry (separated by \n).
- name: my-system-env
  env: env.txt

Other options like files will load the whole content of the file into a single value of the ConfigMap.