Multiple Prometheus Instances with Single Prometheus-Operator

Question

We are looking to implement a monitoring and alerting solution where we would like to give every functional unit there own prometheus instance.

Currently we are running it via prometheus-operator with single prometheus instance but now we need to scale it to multiple prometheus instances with a single operator.

Can someone please provide me to the right direction?

Answer 1

This is quite straightforward: just deploy multiple Prometheus objects to your cluster with different configurations. prometheus-operator will manage any number of prometheus instances.

Added after comment clarification:

prometheus-operator is a piece of software running in your cluster that looks for new/changed/deleted objects with kind: Prometheus (and some others) and creates "regular" k8s objects based on Prometheus entities config.

You can absolutely install multiple instances of prometheus-operator chart (I'm assuming you mean chart that was named promethues-operator and now is renamed to https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack)

At any time there should be only one instance of prometheus-operator (software, not chart) running, so just make sure that any additional releases do not install prometheus-operator itself (software, not the chart). It is available as a values parameter to pass to helm:

prometheusOperator:
  enabled: false