run cron job only when machine is idle (linux)

Question

How can I run a cron job(bash script) only when CPU idle >50%?

I can get cpu idle from TOP

top -b -d 00.10 -n 3 |grep ^Cpu
Cpu(s): 0.3%us, 0.3%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

my current setup is:

crontab
0,15,30,45 * * * * /usr/bin/php /home/user/batchprocess.php
# I could use a bash script here to call PHP, if it is a good solution.

And I have the PHP script check for CPU idle:

batchprocess.php
proc_nice(10);
// wait for CPU idle
do{
    $cpu_stat = exec('top -b -d 00.10 -n 3 |grep ^Cpu');    
    $tmp = stristr($cpu_stat,'%id',TRUE);
    $cpuidle = trim(substr($tmp,strrpos($tmp,',')+1));
}while($cpuidle<$min_cpuidle);
// do actual processing here

The problem with my current method is it starts the program regardless of CPU utilization. And the while loop running TOP feels not efficient. I wish it only starts when CPU idle>50

Some additional info:

• Centos 6.2, PHP5.3

• I have a few EC2 instances that never shuts down, so I want to utilize their processing power when idle. But never load the server heavily. (redundancy DB instance, Development instance, NAT instance)

• I know EC2 auto scaling, spot instance. Just want to use the extra capacity.

• the background job is image compression (CPU intensive, not much I/O or network).

Any suggestion is welcome. thanks in advance!

---

Based on the input below, I realized that "nice" is a better solution in my case. I should re-align my objective to minimize impact to the server instead of tracing CPU utilization.

So the new setup is:

crontab
0,15,30,45 * * * * nice -20 /usr/bin/php /home/user/batchprocess.php

And PHP script:

batchprocess.php
if ($cpuidle < 50) 
    exit(0);
// do actual processing here

I'll test it and post back my finding.

---

report back: I have put this code through DEV/PRD, it works quite well. It does not solve the problem of TOCTOU but is good enough for now.

Answer 1

This is a typical case of TOCTOU - you check if the system is idle, and start your process - but after your check, as or before your process starts, something else caused another process in the system to kick off and you still load the system more than necessary.

The "correct" way to do this is to give your process a low priority, using the nice command. By the way, your loop to check cpu usage will use 100% cpu, so it probably wouldn't work unless THE FIRST time you check it's idle.

You already have a "proc_nice(10)", so that should do the job. I don't see any point in spending effort in identifying whether the system is busy or not.

If you watned to, at suitable points in your code, you could do something like:

 if (check_cpu_usage() > 50%) sleep(1second); 

But I'm not sure if that's much use - if the system is busy, a "nice" process won't receive much CPU-time, and thus won't compete with other processes that run at higher priority.

Answer 2

If you simply wish to have your script executed whenever the system load is e.g. 2.0 or less, you could use a shell script like this:

#!/bin/sh
LOAD=`cat /proc/loadavg | cut -d" " -f1`
THRESHOLD=2.0
if [ $(bc <<< "$LOAD <= $THRESHOLD") -eq 1 ]; then
    $@
fi

Save it as e.g. /usr/local/bin/if-idle, and stick if-idle in front of your command in your crontab file.