Difference between Kubernetes prometheues operator vs helm chart

Question

Can anyone explain in simple terms what is the difference between deploying Prometheus through Kubernetes operator and Prometheus helm chart or manifest file? The question is not just pertaining to Prometheus alone but in general

Answer 1

In general Helm is like a package manager for kubernetes whereas operator is a controller which manages the life cycle of particular kubernetes resource(s).

Operator is a combination of "Custom Resource" and a "Controller" managing the custom resource. There are many native kubernetes resources like pods, deployments, statefulsets, services etc. Each of resources have a corresponding controller process running (currently inside the controller manager). When you create/modify instance of one of the native kubernetes resources using kubectl apply for example, the new/updated object will stored in etcd and api-server will notify the controller about change. The specification in etcd is called the desired state. The job of the controller is to identify the actual state of the cluster and to reconcile it with desired state.

Similar to native kubernetes resources, kubernetes also allows you to create new types of resources using "Custom Resource Definitions". You can also write a controller whose job is to reconcile the your new custom resource. This is called an operator. Usually operators are written to manage the deployment, lifecycle of applications on kubernetes. So Prometheus guys have created a custom resource called Prometheus (using which you can express the desired state of a Prometheus cluster) and a controller which reconciles the desired state with actual state. This controller is called Prometheus operator.

Helm is package manager for kubernetes resources which are generally declared as yaml based manifests. Helm allows you to template these yaml manifests so that you can substitute different values for these templated snippets when deploying the resources in an actual cluster. You can deploy various kubernetes resources including all native resources like pods, services, deployments etc. Operators also run as pods on kubernetes cluster, so helm can be used to deploy operators as well.

You can read about custom resources and custom controllers here - https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#custom-resources

Answer 2

Both prometheus and the prometheus operator can be installed via helm charts. For installing prometheus operator you have this chart while for deploying just prometheus you have this chart

So your question would rather be: what's the difference between installing an operator vs installing the application directly.

The answer is that the operator does more things for you, one example being that it makes service discovery easier: in order to get data from a service you must usually add an annotation to it, altering the target service, while with the operator you just create a ServiceMonitor resource.