Kubectl - How to Read Ingress Hosts from Config Variables?

Question

I have a ConfigMap with a variable for my domain:

apiVersion: v1
kind: ConfigMap
metadata:
  name: config
data:
  MY_DOMAIN: mydomain.com

and my goal is to use the MY_DOMAIN variable inside my Ingress config

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myingress
spec:
  tls:
    - hosts:
⮕       - config.MY_DOMAIN
      secretName: mytls
  rules:
⮕   - host: config.MY_DOMAIN
      http:
        paths:
          - backend:
              serviceName: myservice
              servicePort: 3000

But obviously the config above is not valid. So how can this be achieved?

Answer 1

The configMapRef and secretMapRef for the envFrom and valueFrom functions are only available for environment variables which means they cannot be used in this context. The desired functionality is not available in vanilla Kubernetes as of 1.18.0.

However, it can be done. Helm and Kustomize are probably the two best ways to accomplish this but it could also be done with sed or awk. Helm is a templating engine for Kubernetes manifests. Meaning, you create generic manifests, template out the deltas between your desired manifests with the generic manifests by variables, and then provide a variables file. Then, at runtime, the variables from your variables file are automatically injected into the template for you.

Another way to accomplish this is why Kustomize. Which is what I would personally recommend. Kustomize is like Helm in that it deals with producing customized manifests from generic ones, but it doesn't do so through templating. Kustomize is unique in that it performs merge patches between YAML or JSON files at runtime. These patches are referred to as Overlays so it is often referred to as an overlay engine to differentiate itself from traditional templating engines. Reason being Kustomize can be used with recursive directory trees of bases and overlays. Which makes it much more scalable for environments where dozens, hundreds, or thousands of manifests might need to be generated from boilerplate generic examples.

So how do we do this? Well, with Kustomize you would first define a kustomization.yml file. Within you would define your Resources. In this case, myingress:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
    - myingress.yml

So create a example directory and make a subdirectory called base inside it. Create ./example/base/kustomization.yml and populate it with the kustomization above. Now create a ./example/base/myingress.yml file and populate it with the example myingress file you gave above.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myingress
spec:
  tls:
    - hosts:
      - config.MY_DOMAIN
      secretName: mytls
  rules:
     - host: config.MY_DOMAIN
        http:
          paths:
            - backend:
                serviceName: myservice
                servicePort: 3000

Now we need to define our first overlay. We'll create two different domain configurations to provide an example of how overlays work. First create a ./example/overlays/domain-a directory and create a kustomization.yml file within it with the following contents:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
    - ../../../base/

patchesStrategicMerge:
    - ing_patch.yml

configMapGenerator:
    - name: config_a
      literals:
        - MY_DOMAIN='domain_a'

At this point we have defined ing_patch.yml and config_a in this file. ing_patch.yml will serve as our ingress Patch and config_a will serve as our configMap. However, in this case we'll be taking advantage of a Kustomize feature known as a configMapGenerator rather than manually creating configMap files for single literal key:value pairs.

Now that we have done this, we have to actually make our first patch! Since the deltas in your ingress are pretty small, it's not that hard. Create ./example/overlays/domain_a/ing_patch.yml and populate it with:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myingress
spec:
  tls:
    - hosts:
      - domain.a.com
  rules:
     - host: domain.a.com

Perfect, you have created your first overlay. Now you can use kubectl or kustomize to generate your resultant manifest to apply to the Kubernetes API Server.

• Kubectl Build: kubectl kustomize ./example/overlays/domain_a
• Kustomize Build: kustomize build ./example/overlays/domain_a

Run one of the above Build commands and review the STDOUT produced in your terminal. Notice how it contains two files, myingress and config? And myingress contains the Domain configuration present in your overlay's patch?

So, at this point you're probably asking. Why does Kustomize exist if Kubectl supports the features by default? Well Kustomize started as an external project initially and the Kustomize binary is often running a newer release than the version available in Kubectl.

The next step is to create a second overlay. So go ahead and cp your first overlay over: cp -r ./example/overlays/domain_a ./example/overlays/domain_b.

Now that you have done that, open up ./example/overlays/domain_b/ing_patch.yml up in a text editor and change the contents to look like so:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myingress
spec:
  tls:
    - hosts:
      - domain.b.com
  rules:
     - host: domain.b.com

Save the file and then build your two separate overlays:

kustomize build ./example/overlays/domain_a
kustomize build ./example/overlays/domain_b

Notice how each generated stream of STDOUT varies based on the patch present in the Overlay directory? You can continue to abstract this pattern by making your Bases the Overlays for other bases. Or by making your Overlays the Bases for other Overlays. Doing so can allow you to scale this project in extremely powerful and efficient ways. Apply them to your API Server if you wish:

kubectl apply -k ./example/overlays/domain_a
kubectl apply -k ./example/overlays/domain_b

This is only the beginning of Kustomize really. As you might have guessed after seeing the configMapGenerator field in the kustomization.yml file for each overlay, Kustomize has a LOT of features baked in. It can add labels to all of your resources, it can override their namespaces or container image information, etc.

I hope this helps. Let me know if you have any other questions.