Accurate calculation of CPU usage given in percentage in Linux?

Question

It's a question which has been asked many times, however there is no well supported answer I could find.

Many people suggest the use of top command, but if you run top once (because you have a script for example collecting Cpu usage every 1 second) it will always give the same Cpu usage result (example 1, example 2).

A more accurate way to calculate CPU usage, is by reading the values from /proc/stat, but most of the answers use only the first 4 fields from /proc/stat to calculate it (one example here).

/proc/stat/ has 10 fields per CPU core as of Linux kernel 2.6.33!

I also found this Accurately Calculating CPU Utilization in Linux using /proc/stat question which is pointing out the same issue, -that most other questions only take into consideration 4 out of the many fields- but still the answer given here starts with "I think" (not certain), and except that, it is only concerned about the first 7 fields (out of 10 in /proc/stat/)

This perl script uses all of the fields to calculate the CPU usage, which again I do not think is correct after some further investigation.

After taking a quick look into the kernel code here, it looks like, for example, guest_nice and guest fields are always increasing together with nice and user (so they should not be included in the cpu usage calculation, since they are included in nice and user fields already)

/*  * Account guest cpu time to a process.  * @p: the process that the cpu time gets accounted to  * @cputime: the cpu time spent in virtual machine since the last update  * @cputime_scaled: cputime scaled by cpu frequency  */ static void account_guest_time(struct task_struct *p, cputime_t cputime,                    cputime_t cputime_scaled) {     u64 *cpustat = kcpustat_this_cpu->cpustat;      /* Add guest time to process. */     p->utime += cputime;     p->utimescaled += cputime_scaled;     account_group_user_time(p, cputime);     p->gtime += cputime;      /* Add guest time to cpustat. */     if (task_nice(p) > 0) {         cpustat[CPUTIME_NICE] += (__force u64) cputime;         cpustat[CPUTIME_GUEST_NICE] += (__force u64) cputime;     } else {         cpustat[CPUTIME_USER] += (__force u64) cputime;         cpustat[CPUTIME_GUEST] += (__force u64) cputime;     } } 

So to sum up, what is an accurate way to calculate the CPU usage in Linux and which fields should be considered in the calculations and how (which fields are attributed to the idle time and which fields to non-idle time)?

Answer 1

According the htop source code, my assumptions looks like they are valid:

(see static inline double LinuxProcessList_scanCPUTime(LinuxProcessList* this) function at LinuxProcessList.c)

// Guest time is already accounted in usertime usertime = usertime - guest;                             # As you see here, it subtracts guest from user time nicetime = nicetime - guestnice;                         # and guest_nice from nice time // Fields existing on kernels >= 2.6 // (and RHEL's patched kernel 2.4...) unsigned long long int idlealltime = idletime + ioWait;  # ioWait is added in the idleTime unsigned long long int systemalltime = systemtime + irq + softIrq; unsigned long long int virtalltime = guest + guestnice; unsigned long long int totaltime = usertime + nicetime + systemalltime + idlealltime + steal + virtalltime; 

And so, from fields listed in the first line of /proc/stat: (see section 1.8 at documentation)

     user    nice   system  idle      iowait irq   softirq  steal  guest  guest_nice cpu  74608   2520   24433   1117073   6176   4054  0        0      0      0 

Algorithmically, we can calculate the CPU usage percentage like:

PrevIdle = previdle + previowait Idle = idle + iowait  PrevNonIdle = prevuser + prevnice + prevsystem + previrq + prevsoftirq + prevsteal NonIdle = user + nice + system + irq + softirq + steal  PrevTotal = PrevIdle + PrevNonIdle Total = Idle + NonIdle  # differentiate: actual value minus the previous one totald = Total - PrevTotal idled = Idle - PrevIdle  CPU_Percentage = (totald - idled)/totald 

Answer 2

The following is a bash script which is based on Vangelis's answer. It produces output like this:

total 49.1803 cpu0 14.2857 cpu1 100 cpu2 28.5714 cpu3 100 cpu4 30 cpu5 25 

Create a file called get_cpu_usage.sh

Run it using the following command: bash get_cpu_usage.sh 0.2

The argument is the number of seconds to measure. In this case it's 200 milliseconds.

The contents are:

#!/bin/sh  sleepDurationSeconds=$1  previousDate=$(date +%s%N | cut -b1-13) previousStats=$(cat /proc/stat)  sleep $sleepDurationSeconds  currentDate=$(date +%s%N | cut -b1-13) currentStats=$(cat /proc/stat)      cpus=$(echo "$currentStats" | grep -P 'cpu' | awk -F " " '{print $1}')  for cpu in $cpus do     currentLine=$(echo "$currentStats" | grep "$cpu ")     user=$(echo "$currentLine" | awk -F " " '{print $2}')     nice=$(echo "$currentLine" | awk -F " " '{print $3}')     system=$(echo "$currentLine" | awk -F " " '{print $4}')     idle=$(echo "$currentLine" | awk -F " " '{print $5}')     iowait=$(echo "$currentLine" | awk -F " " '{print $6}')     irq=$(echo "$currentLine" | awk -F " " '{print $7}')     softirq=$(echo "$currentLine" | awk -F " " '{print $8}')     steal=$(echo "$currentLine" | awk -F " " '{print $9}')     guest=$(echo "$currentLine" | awk -F " " '{print $10}')     guest_nice=$(echo "$currentLine" | awk -F " " '{print $11}')      previousLine=$(echo "$previousStats" | grep "$cpu ")     prevuser=$(echo "$previousLine" | awk -F " " '{print $2}')     prevnice=$(echo "$previousLine" | awk -F " " '{print $3}')     prevsystem=$(echo "$previousLine" | awk -F " " '{print $4}')     previdle=$(echo "$previousLine" | awk -F " " '{print $5}')     previowait=$(echo "$previousLine" | awk -F " " '{print $6}')     previrq=$(echo "$previousLine" | awk -F " " '{print $7}')     prevsoftirq=$(echo "$previousLine" | awk -F " " '{print $8}')     prevsteal=$(echo "$previousLine" | awk -F " " '{print $9}')     prevguest=$(echo "$previousLine" | awk -F " " '{print $10}')     prevguest_nice=$(echo "$previousLine" | awk -F " " '{print $11}')          PrevIdle=$((previdle + previowait))     Idle=$((idle + iowait))      PrevNonIdle=$((prevuser + prevnice + prevsystem + previrq + prevsoftirq + prevsteal))     NonIdle=$((user + nice + system + irq + softirq + steal))      PrevTotal=$((PrevIdle + PrevNonIdle))     Total=$((Idle + NonIdle))      totald=$((Total - PrevTotal))     idled=$((Idle - PrevIdle))      CPU_Percentage=$(awk "BEGIN {print ($totald - $idled)/$totald*100}")      if [[ "$cpu" == "cpu" ]]; then         echo "total "$CPU_Percentage     else         echo $cpu" "$CPU_Percentage     fi done