Send alert email if CPU usage is continuously higher than a certain amount

Question

In a Linux / Unix server when the CPU usage go above a threshold value it need to send out a email alert. Propose a way to do it through cron tab and shell scripting.

Answer 1

This can be done through following shell script and a frequent cron job.

cpu_monitor.sh

CPU=$(sar 1 5 | grep "Average" | sed 's/^.* //')

if [ $CPU -lt 20 ]
then
   cat mail_content.html | /usr/lib/sendmail -t
else
   echo "Normal"
fi

mail_content.html

From: [email protected]
To: [email protected]
Subject: Subject of the mail
Mime-Version: 1.0
Content-Type: text/html

<h1>CPU usage increased heigh</h1>

Here the script will take the CPU ideal percentage for each 1 seconds. And 5 samples will be taken. Then average of that ideal percentage will be passed to variable CPU. When the ideal goes below the 20% mail will be send out.

We can setup the cron with 5 minute duration.

*/5 * * * * cd /full/path/to/script/; ./cpu_monitor.sh;

Answer 2

I would use the uptime command that displays (at the end) the average cpu usage for the last 15 minutes. This should be a reasonable time range in order to avoid short spikes in CPU usage. The uptime command shows the number of processes in a runnable state. If they are less than the CPU cores, the machine is not CPU bound, while if they are more than the CPU cores, the machines will start suffering. Let's say we agree that having the number of processes more than 2 times the number of cores is the level we want to be alerted.

# get the number of CPU cores using /proc/cpuinfo
#
ncpu=$(egrep -c '^processor' /proc/cpuinfo)

#
# get the last number (only integer part) of uptime output
#
load=$(LC_ALL=C uptime  | sed -e 's/.*average: .* \([0-9]*\)\.[0-9][0-9]$/\1/')

#
# divide the $load number by the $ncpu and check if greather than alert level
#
if [ $(($load/$cpu)) -gt 2 ]
then
   # send an e-mail to [email protected].
   echo -e  "\nCPU usage is greather than twice the number of CPU cores for more than 15 minutes.\n" \
   | fold -s -w75 | mailx -s "CPU alert for $(hostname)" [email protected]
fi

It may be run from crontab adding this text as /etc/cron.d/simplecpucheck:

[email protected]
10,40 * * * * root /path/name/of/shell-script

Please note that I prefer to send e-mail from the shell script instead of crontab daemon since I may use a subject line more informative. I still leave the email address in crontab file in order to receive any other error (mainly syntax errors in my script and commands not found).