Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Elegant solution to implement timeout for bash commands and functions

Tags:

bash

I wrote a function to run commands, which takes two args 1st a command 2nd timeout in seconds:

#! /bin/bash

function run_cmd {
    cmd="$1"; timeout="$2"
    grep -qP "^\d+$" <<< "$timeout" || timeout=10

    stderrfile=$(readlink /proc/$$/fd/2)
    exec 2<&-

    exitfile=/tmp/exit_$(date +%s.%N)
    (eval "$cmd";echo $? > $exitfile) &

    start=$(date +%s)
    while true; do
        pid=$(jobs -l | awk '/Running/{print $2}')
        if [ -n "$pid" ]; then
            now=$(date +%s)
            running=$(($now - $start))
            if [ "$running" -ge "$timeout" ];then
                kill -15 "$pid"
                exit=1
            fi
            sleep 1
        else
            break
        fi

    done 
    test -n "$exit" || exit=$(cat $exitfile)
    rm $exitfile
    exec 2>$stderrfile              
    return "$exit"
}


function sleep5 {
    sleep 5
    echo "I slept 5"
    return 2
}

run_cmd sleep5 "6" 
run_cmd sleep5 "3"
echo "hi" >&2 

The function works fine but I am not sure it's an elegant solution, I would like to know about alternatives for the following

  1. I am having to store exit status on a file: (eval "$cmd";echo $? > $exitfile)
  2. I am closing and reopening STDERR: exec 2<&- and exec 2>$stderrfile

I am closing STDERR because I couldn't avoid the message when killing the command:

test.sh: line 3: 32323 Terminated ( eval "$cmd"; echo $? > $exitfile )

PS: I am aware of timeout and expect but they wouldn't work for functions.

like image 616
Tiago Lopo Avatar asked Jun 25 '14 15:06

Tiago Lopo


People also ask

What is bash timeout?

The timeout command stops an executed process after the timeout period: $ timeout 1s bash -c 'for((;;)); do :; done' $ echo $? 124. Here, we run an endless loop. We set a timeout of one second before timeout should kill the process. Importantly, note the exit code.

How do I put sleep in a bash script?

How to Use the Bash Sleep Command. Sleep is a very versatile command with a very simple syntax. It is as easy as typing sleep N . This will pause your script for N seconds, with N being either a positive integer or a floating point number.


3 Answers

Perhaps this suits your needs. I changed the call signature to make it possible to avoid using eval.

# Usage: run_with_timeout N cmd args...
#    or: run_with_timeout cmd args...
# In the second case, cmd cannot be a number and the timeout will be 10 seconds.
run_with_timeout () { 
    local time=10
    if [[ $1 =~ ^[0-9]+$ ]]; then time=$1; shift; fi
    # Run in a subshell to avoid job control messages
    ( "$@" &
      child=$!
      # Avoid default notification in non-interactive shell for SIGTERM
      trap -- "" SIGTERM
      ( sleep $time
        kill $child 2> /dev/null ) &
      wait $child
    )
}

Example, showing exit status:

$ sleep_and_exit() { sleep ${1:-1}; exit ${2:-0}; }

$ time run_with_timeout 1 sleep_and_exit 3 0; echo $?

real    0m1.007s
user    0m0.003s
sys     0m0.006s
143

$ time run_with_timeout 3 sleep_and_exit 1 0; echo $?

real    0m1.007s
user    0m0.003s
sys     0m0.008s
0

$ time run_with_timeout 3 sleep_and_exit 1 7; echo $?

real    0m1.006s
user    0m0.001s
sys     0m0.006s
7

As shown, the exit status of run_with_timeout will be the exit status of the executed command unless it was killed by the timeout, in which case it will be 143 (128 + 15).

Note: If you set a large timeout and/or have a forkbomb running, you might recycle pids fast enough that the kill-child kills the wrong process.

like image 53
rici Avatar answered Nov 04 '22 21:11

rici


If you want to control functions, you may use a trap handler (like in C)

$ trap 'break' 15
$ echo $$; while :; do :; done; echo 'endlessloop terminated'
5168
endlessloop terminated
$

If you type kill -15 5168 in another shell, the program interrupts and prints endlessloop terminated

If you are spawning a subprocess, please take care about four additional things

  1. if the subprocess ends long before the sleep, it results in a long sleep process. So it is better to keep the sleep short and keep on checking multiple times. For example, it is better to do 360 sleeps of 10s than to sleep 3600s = 1hour. Because the sleep may fill up your process table up to the limit. (Or you have to kill the sleep as soon as the $cmd finishes.)

  2. if the process does not react on a normal kill, you may want to add an additional kill -9 a few seconds afterwards.

  3. if you need the return value of the process, then you have to extend you program using a wrapper, which delivers the return value to a file/fifo.

  4. if you need the stdout/stderr output of the process, ... file/fifo.

All these things are covered by the C program timelimit.

http://devel.ringlet.net/sysutils/timelimit/

$ timelimit
timelimit: using defaults: warntime=3600, warnsig=15, killtime=120, killsig=9
timelimit: usage: timelimit [-pq] [-S ksig] [-s wsig] [-T ktime] [-t wtime] command

This program has a few benefits:

  • it checks, if the process is still running and has not exited while sleeping for kill
  • if sends a soft killsignal first and a hard -9 signal if this does not work
  • it propagates (Option -p) the returnlevel ($?) so you may use it for your purpose.
like image 30
lx42.de Avatar answered Nov 04 '22 20:11

lx42.de


I believe I've got a elegant solution based on @rici answer (which I accepted), and decided I would share the end result, I also added a retry function which was the real goal.

function run_cmd { 
    cmd="$1"; timeout="$2";
    grep -qP '^\d+$' <<< $timeout || timeout=10

    ( 
        eval "$cmd" &
        child=$!
        trap -- "" SIGTERM 
        (       
                sleep $timeout
                kill $child 
        ) > /dev/null 2>&1 &     
        wait $child
    )
}

function retry { 
        cmd=$1; timeout=$2; tries=$3; interval=$4
        grep -qP '^\d+$' <<< $timeout || timeout=10
        grep -qP '^\d+$' <<< $tries || tries=3 
        grep -qP '^\d+$' <<< $interval || interval=3
        for ((c=1; c <= $tries; c++)); do
                run_cmd "$cmd" "$timeout" && return
                sleep $interval
        done    
        return 1
}

The retry function accepts 4 args:

  1. The command
  2. Timeout
  3. Attempts
  4. Interval

It can be executed as below:

retry "some_command_or_function arg1 arg2 .." 5 2 10

like image 1
Tiago Lopo Avatar answered Nov 04 '22 19:11

Tiago Lopo