%CPU column and load in top

Question

One of my c++ application shows very high %CPU in top, but the load is quite low. myapp always takes around 30%, showing at the very top of the "top" command, but the load is always like 0.00, so I'm confused what are the differences between load and %CPU column?

top - 14:09:54 up 62 days,  2:52,  1 user,  load average: 0.00, 0.00, 0.00
Tasks: 124 total,   1 running, 123 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.0%sy,  0.0%ni, 99.7%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni, 99.3%id,  0.3%wa,  0.0%hi,  0.3%si,  0.0%st
Mem:   2054824k total,  1440364k used,   614460k free,     4644k buffers
Swap:  4194296k total,    16604k used,  4177692k free,   610784k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                         
14774 root      20   0 1201m 9416 1856 S 28.3  0.5 376:13.50 myapp                                                                                        
14746 mysql     20   0 2012m 157m 3888 S  0.3  7.8   5:01.08 mysqld                                                                                          

Can anyone tell me understand the differences between %CPU and load?

Thank you in advance!

Answer 1

%CPU shown in for example top is intantaneous, i.e. what is being used at that time, while the load shown by uptime is an average:

It conventionally appears in the form of three numbers which represent the system load during the last one-, five-, and fifteen-minute periods.

from Wikipedia.

Over time these should converge, but while %CPU is 100% maximum, load can be higher than 1 when though there are processes waiting. For example:

For example, one can interpret a load average of "1.73 0.50 7.98" on a single-CPU system as:

during the last minute, the CPU was overloaded by 73% (1 CPU with 1.73 runnable processes, so that 0.73 processes had to wait for a turn)

during the last 5 minutes, the CPU was underloaded 50% (no processes had to wait for a turn)

during the last 15 minutes, the CPU was overloaded 698% (1 CPU with 7.98 runnable processes, so that 6.98 processes had to wait for a turn)

Update:

I only now noticed that your process was using 28% CPU while top reported both CPUs to be >99% idle. I was not able to reproduce this on my Linux system (Ubuntu 12.04, Intel i7-3770K quadcore HT), see screenshots below.

Normal setup, no load, showing CPU usage averaged over all cores:

top - 18:10:04 up  7:50,  2 users,  load average: 0.00, 0.05, 0.15
Tasks: 157 total,   1 running, 156 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni, 99.9%id,  0.0%wa,  0.0%hi,  0.0%si, 0.0%st
Mem:  16528224k total,  1124956k used, 15403268k free,   148772k buffers
Swap: 15624188k total,        0k used, 15624188k free, 670460k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    7022 root      20   0     0    0    0 S    0  0.0   0:00.14 kworker/0:1        
    1 root      20   0  3640 2060 1324 S    0  0.0   0:01.23 init               
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd           
    3 root      20   0     0    0    0 S    0  0.0   0:00.14 ksoftirqd/0        
    6 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/0

Full load on one core (running yes > /dev/null), showing CPU usage averaged over all cores:

top - 18:11:58 up  7:52,  2 users,  load average: 0.11, 0.07, 0.15
Tasks: 157 total,   2 running, 155 sleeping,   0 stopped,   0 zombie
Cpu(s): 12.5%us,  0.0%sy,  0.0%ni, 87.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16528224k total,  1124880k used, 15403344k free,   148824k buffers
Swap: 15624188k total,        0k used, 15624188k free,   670472k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
 8254 tim       20   0  4188  280  228 R  100  0.0   0:04.88 yes                
    1 root      20   0  3640 2060 1324 S    0  0.0   0:01.23 init               
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd           
    3 root      20   0     0    0    0 S    0  0.0   0:00.15 ksoftirqd/0        
    6 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/0        

Same as 2), but showing per-core CPU usage:

top - 18:13:47 up  7:54,  2 users,  load average: 0.86, 0.36, 0.24
Tasks: 157 total,   2 running, 155 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16528224k total,  1124756k used, 15403468k free,   148840k buffers
Swap: 15624188k total,        0k used, 15624188k free,   670472k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
 8254 tim       20   0  4188  280  228 R  100  0.0   1:53.98 yes                
    1 root      20   0  3640 2060 1324 S    0  0.0   0:01.23 init               
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd           
    3 root      20   0     0    0    0 S    0  0.0   0:00.15 ksoftirqd/0        
    6 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/0        

N.B.: I did note that the first statistics top shows do not show the 100% cpu usage for yes, similar to what you have. Perhaps that was the problem?