Helm Test best practices

Question

Just to clarify.

1. I am not talking about Helm chart best practices. I can read them by myself over here: https://helm.sh/docs/chart_best_practices/
2. I am not talking about Terratest.

I am interested in Helm Test you can read more about it over here https://helm.sh/docs/chart_tests/#chart-tests .

Unfortunately, the documentation doesn't provide you a lot of information and that is partly the reason for this question.

What we have:

1. We can have a test defined in a Chart by the following path

our-app/templates/tests/test-our-app.yaml

2. As far as I can understand and according to documentation, it is not a unit test for chart but more like a smoke test or integration test. Documentation: "validate that your chart works as expected when it is installed". This is exactly what I need.
3. In order to run the test https://helm.sh/docs/chart_tests/#steps-to-run-a-test-suite-on-a-release. We need to install our-app Chart

    $ helm install our-app

and then we can test that image

    $ helm test release-with-our-app

Things I would like to clarify:

1. Should we have two separate docker images or one for application and tests?...

   a). In the case of it being single one image for both application code and test code then we are adding code for the test/tests all on that image, which makes it bigger and additionally we need to put dependencies that are required by tests on that image. To me, this solution seems to be wrong.

   b). The case where we have a separate image for tests makes more sense because our app docker image is free of any unnecessary dependencies. What's more, in the second solution we don't really care about the size of the docker image with the test(s) because it is supposed to be a short-living image that shuts down when tests on it are finished.

   Is my assumption correct and we should have a separate image for these tests?

2. In case if have two separate images, one for code another for tests. How do we marry them with each other? Does it mean that we have to pass a Build/Release number to helm test command so it is able to pull down the correct image for tests?

3. Also in case if we have two separate images. Should I specify the test run in a docker file for tests? for example :

   CMD ["sh", "-c", "gradle test -Denvironment=$ENVIRONMENT"]

Or should it be inside chart

   our-app/templates/tests/test-our-app.yaml

For example test.yaml from mysql repo:

https://github.com/helm/charts/blob/master/stable/mysql/templates/tests/test-configmap.yaml

4. It looks like environments are not mentioned at all in the documentation. My question is what is the best/recommended way to pass environment name to the chart so tests know which endpoints to hit etc? Should it be Chart args How to pass dynamic arguments to a helm chart that runs a job ?

Answer 1

an example you can find in mysql chart by the following link https://github.com/helm/charts/tree/master/stable/mysql/templates

Answering my own question:

1. We made a separate docker image for helm tests due to the following reasons:
   1.1 When helm smoke docker image is separate that makes it free of any constraints of the main app docker image. So you can install any libraries and avoid following some security guidelines that main app has to follow.
   1.2 There is no restriction on the size of a docker image for helm test becasue it is short living image , the image of helm test is a live for a duration of tests inside.
2. We marry them via Chart version. So the image of main application has always the same version as the helm tests docker image.

We don't have to pass anything, when you run "helm test yourapp", helm looks for successfully deployed version of yourapp and runs helm test with that version.

3. When you run the "helm test yourapp" command. Helm looks in file
yourapp/templates/tests/smoke-test.yaml
calls command that is specified by the following path:
spec/containers/command: ["sh", "-c", "your command"] plz have a look on the screenshot or go to git repo of mysql chart https://github.com/helm/charts/tree/master/stable/mysql/templates

4. The best way to pass data and Env specific data is passing them via config maps, env variables, and secrets as Hanx unanswered. If it is stored in such a way then you can populate such data via helm values files and it will be easy to update. We found that having env specific data inside image is a bad solution because if for example a URL/host changes in some environment we need to create a new build instead of just updating a values file and performing deploy.

Answer 2

Just starting this topic:

In my opinion it depends on your needs.

For the 1,2,3 - as I understood "helm test" is very flexible and it can perform any additional tests depends on your needs.

You can use different images and different approaches or one image with different arguments but sometimes probably it can be impossible due to docker entrypoint - please see commands and arguments.

According to the 4 question - in my opinion you should consider using configmaps and secrets to define and expose env variables into your pods.

Additional resources "helm test":

• Integration testing deployed services using “helm test”
• How to use helm chart test to do integration tests?

Hope this help.