Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why Prometheus pod pending after setup it by helm in Kubernetes cluster on Rancher server?

Installed Rancher server and 2 Rancher agents in Vagrant. Then switch to K8S environment from Rancher server.

On Rancher server host, installed kubectl and helm. Then installed Prometheus by Helm:

helm install stable/prometheus

Now check the status from Kubernetes dashboard, there are 2 pods pending: enter image description here

It noticed PersistentVolumeClaim is not bound, so aren't the K8S components been installed default with Rancher server?

(another name, same issue) enter image description here

Edit

> kubectl get pvc
NAME                                   STATUS    VOLUME    CAPACITY   
ACCESSMODES   STORAGECLASS   AGE
voting-prawn-prometheus-alertmanager   Pending                                                     6h
voting-prawn-prometheus-server         Pending                                                     6h
> kubectl get pv
No resources found.

Edit 2

$ kubectl describe pvc voting-prawn-prometheus-alertmanager
Name:          voting-prawn-prometheus-alertmanager
Namespace:     default
StorageClass:
Status:        Pending
Volume:
Labels:        app=prometheus
               chart=prometheus-4.6.9
               component=alertmanager
               heritage=Tiller
               release=voting-prawn
Annotations:   <none>
Capacity:
Access Modes:
Events:
  Type    Reason         Age                From                         Message
  ----    ------         ----               ----                         -------
  Normal  FailedBinding  12s (x10 over 2m)  persistentvolume-controller  no persistent volumes available for this claim and no storage class is set

$ kubectl describe pvc voting-prawn-prometheus-server
Name:          voting-prawn-prometheus-server
Namespace:     default
StorageClass:
Status:        Pending
Volume:
Labels:        app=prometheus
               chart=prometheus-4.6.9
               component=server
               heritage=Tiller
               release=voting-prawn
Annotations:   <none>
Capacity:
Access Modes:
Events:
  Type    Reason         Age                From                         Message
  ----    ------         ----               ----                         -------
  Normal  FailedBinding  12s (x14 over 3m)  persistentvolume-controller  no persistent volumes available for this claim and no storage class is set
like image 936
online Avatar asked Mar 08 '23 15:03

online


1 Answers

I had same issues as you. I found two ways to solve this:

  • edit values.yaml under persistentVolumes.enabled=false this will allow you to use emptyDir "this applies to Prometheus-Server and AlertManager"

  • If you can't change values.yaml you will have to create the PV before deploying the chart so that the pod can bind to the volume otherwise it will stay in the pending state forever

like image 77
Abdel Hegazi Avatar answered Apr 05 '23 20:04

Abdel Hegazi