Kill all R processes that hang for longer than a minute

Question

I use crontask to regularly run Rscript. Unfortunately, I need to do this on a small instance of aws and the process may hang, building more and more processes on top of each other until the whole system is lagging.

I would like to write a crontask to kill all R processes lasting longer than one minute. I found another answer on Stack Overflow that I've adapted that I think would solve the problem. I came up with;

if [[ "$(uname)" = "Linux" ]];then killall --older-than 1m "/usr/lib/R/bin/exec/R --slave --no-restore --file=/home/ubuntu/script.R";fi

I copied the task directly from htop, but it does not work as I expect. I get the No such file or directory error but I've checked it a few times.

I need to kill all R processes that have lasted longer than a minute. How can I do this?

Answer 1

You may want to avoid killing processes from another user and try SIGKILL (kill -9) after SIGTERM (kill -15). Here is a script you could execute every minute with a CRON job:

#!/bin/bash

PROCESS="R"
MAXTIME=`date -d '00:01:00' +'%s'`

function killpids()
{
    PIDS=`pgrep -u "${USER}" -x "${PROCESS}"`

    # Loop over all matching PIDs
    for pid in ${PIDS}; do
        # Retrieve duration of the process
        TIME=`ps -o time:1= -p "${pid}" |
              egrep -o "[0-9]{0,2}:?[0-9]{0,2}:[0-9]{2}$"`

        # Convert TIME to timestamp
        TTIME=`date -d "${TIME}" +'%s'`

        # Check if the process should be killed
        if [ "${TTIME}" -gt "${MAXTIME}" ]; then
            kill ${1} "${pid}"
        fi
    done
}

# Leave a chance to kill processes properly (SIGTERM)
killpids "-15"
sleep 5

# Now kill remaining processes (SIGKILL)
killpids "-9"

Answer 2

Why imply an additional process every minute with cron? Would it not be easier to start R with timeout from coreutils, the processes will then be killed automatically after the time you chose.

timeout [option] duration command [arg]…