Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

kubectl top node `error: metrics not available yet` . Using metrics-server as Heapster Depricated

Kubernetes not able to find metric-server api.I am using Kubernetes with Docker on Mac. I was trying to do HPA from following example [https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/]. However, when I execute command kubectl get hpa, My target still was unknown. Then I tried, kubectl describe hpa. Which gave me error like below:

  Name:                                                  php-apache
  Namespace:                                             default
  Labels:                                                <none>
  Annotations:                                           <none>
  CreationTimestamp:                                     Sun, 07 Oct 2018 12:36:31 -0700
  Reference:                                             Deployment/php-apache
  Metrics:                                               ( current / target )
    resource cpu on pods  (as a percentage of request):  <unknown> / 5%
  Min replicas:                                          1
  Max replicas:                                          10
  Conditions:
    Type           Status  Reason                   Message
    ----           ------  ------                   -------
    AbleToScale    True    SucceededGetScale        the HPA controller was able to get the target's current scale
    ScalingActive  False   FailedGetResourceMetric  the HPA was unable to compute the replica count: unable to get metrics for resource cpu: no metrics returned from resource metrics API
  Events:
    Type     Reason                        Age               From                       Message
    ----     ------                        ----              ----                       -------
    Warning  FailedComputeMetricsReplicas  1h (x34 over 5h)  horizontal-pod-autoscaler  failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
    Warning  FailedGetResourceMetric       1m (x42 over 5h)  horizontal-pod-autoscaler  unable to get metrics for resource cpu: no metrics returned from resource metrics API

I am using [https://github.com/kubernetes-incubator/metrics-server] as suggested in Kubernetes documentation. I also tried doing same just using Minikube. But that also didn't work.

Running kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes outputs :

{
  "kind": "NodeMetricsList",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes"
  },
  "items": []
}
like image 387
Vivek Avatar asked Oct 08 '18 01:10

Vivek


People also ask

How do I enable metrics server in Minikube?

A note to minikube users Metrics Server is available as one of the plugins. Please execute minikube addons enable metrics-server and kubectl -n kube-system rollout status deployment metrics-server commands instead of those below.

How to check metrics from pod in Kubernetes?

You can also view these metrics using the kubectl top command. The metrics-server uses the Kubernetes API to track nodes and pods in your cluster. The metrics-server queries each node over HTTP to fetch metrics. The metrics-server also builds an internal view of pod metadata, and keeps a cache of pod health.

What is metrics in Kubernetes?

Kubernetes metrics help you ensure all pods in a deployment are running and healthy. They provide information such as how many instances a pod currently has and how many were expected. If the number is too low, your cluster may run out of resources.


2 Answers

Solution(if using Minikube):

Changed context of Kubernetes to Minikube.

Enabled metrics-server and Disabled heapster in minikube.

minikube addons disable heapster

minikube addons enable metrics-server

Deploy metrics-server in your cluster using the following steps:

git clone https://github.com/kubernetes-incubator/metrics-server.git

cd metrics-server

kubectl create -f deploy/1.7/ (if Kubernetes version 1.7)

OR

kubectl create -f deploy/1.8+/(if Kubernetes version 1.8+)

Start minikube dashboad and minikube service [your service].

Try kubectl top node.

I found this (https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/) resource helpful.

like image 191
Vivek Avatar answered Sep 21 '22 15:09

Vivek


Use official metrics server - https://github.com/kubernetes-sigs/metrics-server

If you use one master node, run this command to create the metrics-server:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

If you have HA (High availability) cluster, use this:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/high-availability.yaml

Then use can use kubectl top nodes or kubectl top pods -A and get something like:

NAME          CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
like image 45
Akhil Manepalli Avatar answered Sep 17 '22 15:09

Akhil Manepalli