Alert if file grows more than a certain size in 5 minutes

Question

I'm trying to write a script to check how much a file has grown in the last 5 minutes, and write out to a log if it grows more than 20MB in that time.

So far I've managed to write this;

output="$HOME/log/logsize.output"                       # This file is where the values are written to before being compared
currentsize=$(stat '-c%s' "$HOME/log/input.log")        # This is the current size of the log file

    echo $currentsize >> $output

oldsize=$(sed 'x;$!d' < "$output")                      # Sets the previous reading as "oldsize" by pulling the second to last line of $output
difference=$(("$currentsize" - "$oldsize"))             # This is the difference in size between the current and previous readings


if $difference > "1999999"                              # Checks the difference, if greater than 1999999 write an error.
    then
    echo "Warning! Log File has exceeded the expected rate of growth. Please investigate." > "$HOME/log/logalert.log"
else
    echo "Log File is within expected parameters" > "$HOME/log/logalert.log"
fi

When I run this script this is the output I recieve;

line 23: "2910" - "2910": syntax error: operand expected (error token is ""2910" - "2910"")

Solved!

Here's what I did in the end to get this working

#!/bin/bash
#########################################################################
# Author - Jack Arnold
#
# Last Updated: 20.02.18
#########################################################################
#
# This script exists to periodically check the file size of a log file.
# If this file has grown 20MB or more since the last loop of this script
# it will write out an alert to ~/logs/logsize.log
#
#########################################################################


# Variables for the script.

output="$HOME/log/logsize.output"                       # This file is where the values are written to before being compared
currentsize=$(stat '-c%s' "$HOME/log/input.log")        # This is the current size of the log file

    echo "$currentsize" >> "$output"

oldsize=$(sed 'x;$!d' < "$output")                      # Sets the previous reading as "oldsize" by pulling the second to last line of $output
difference=$((currentsize - oldsize))                   # This is the difference in size between the current and previous readings


if [[ $difference > "1999999" ]]                        # Checks the difference, if greater than 1999999 write an error.
    then
    echo "Warning! Log File has exceeded the expected rate of growth. Please investigate." > "$HOME/log/logalert.log"
else
    echo "Log File is within expected parameters" > "$HOME/log/logalert.log"
fi

Answer 1

The first thing I noticed was that in the very first line, the variable assignment (output="~/log/logsize.output") would cause problems as ~ is not expanded in quotes. However, there are a lot more errors in this script and I’d advise spending more time learning the basics of shell scripting; I’d suggest Greg’s Wiki as a great starting point.

A while ago, I updated the usage guidance for the bash tag so that it includes the advice to check shell scripts in https://www.shellcheck.net/ which is a fabulous resource. Indeed, Shellcheck warns about the tilde issue and includes the useful suggestion of using $HOME instead of ~. Rather than re-iterating all the issues that Shellcheck would warn you about, I’ll just mention some of the problems that it doesn’t pick up on:

Command substitution

currentsize=(stat '-c%s' "~/log/input.log")

I imagine you intend to use command substitution so that the currentsize variable contains the output of the stat command. This should be written as:

currentsize=$(stat '-c%s' "$HOME/log/input.log")

Arithmetic comparisons

Shellcheck stops processing before it gets to this line but I notice that you are using > as an arithmetic comparison operator:

if (${difference} > 1999999) ;

In POSIX (standard for Unix-like operating systems) shells, these operators are used for input and output redirection – as you’ve done in

echo "Log File is within expected parameters" > "~/log/logalert.log"

The correct operator for arithmetic comparisons in POSIX shells is -gt and the portable way to test this is:

if [ "$difference" -gt 1999999 ]

Note: Shells such as bash and ksh extend POSIX by using < and > for arithmetic comparisons but this only applies within double parentheses. See Comparing integers: arithmetic expression or conditional expression. In Bash, the > can also be used for string comparisons when used with the [[ construct. See Bash Conditional Expressions.

Also: you only really need to quote strings if they contain unusual characters that are specially interpreted by the shell (e.g., spaces result in word-splitting). However, there’s no harm in doing so if you’re already used to it from using other programming languages and you find it to be more readable.

General tips

• Always quote variable expansions (except when you explicitly require word-splitting)
• Use set -x when debugging.
• set -e is also useful to get notified of errors such as attempting to access the contents of a non-existent variable.

Answer 2

I'm just wanted to offer my solution:

#!/bin/bash

output="$HOME/log/logsize.output"
if [ ! -f $HOME/log/logsize.output ]
then
    touch $HOME/log/logsize.output
fi

while [ 1 ]
do
    stat '-c%s' $HOME/log/input.log >> "${output}"
    math=$(tail -n2 "${output}" | tr '\n' '-' | sed 's/.$//g')
# 20971520 bytes is 20 Mbytes. Uncomment this string, and comment mine with -100 argument. Mine string is only for the example.
#   if [ $(($math)) -lt -20971520 ]
    if [ $(($math)) -lt -100 ]
    then
        echo "Attemption! The file have has grown by more then 20Mbytes!"
    fi
# Replace sleep 5 by sleep 600 for 1 per 5 minute check.
sleep 5
done

You can run it as you wish: ./filechange.sh & or in cron (if you wish cron, remove while loop and sleep) How does it works:

$ ls -l
total 4
-rwxr-xr-x 1 sahaquiel sahaquiel 400 Feb 20 15:00 filechange.sh
-rw-r--r-- 1 sahaquiel sahaquiel   0 Feb 20 14:58 input.log
-rw-r--r-- 1 sahaquiel sahaquiel   0 Feb 20 14:59 logsize.output
$ ./filechange.sh & 
[1] 29829
# Adding 150 random numbers in input.log file
[sahaquiel@sahaquiel-PC log]$ i=0; while [ $i -lt 150 ]; do echo $RANDOM >> input.log; i=$(($i + 1)); done
# filechange.sh echo in my shell:
[sahaquiel@sahaquiel-PC log]$ Attemption! The file have has grown by more then 20Mbytes!

$ ls -l
total 12
-rwxr-xr-x 1 sahaquiel sahaquiel 400 Feb 20 15:00 filechange.sh
-rw-r--r-- 1 sahaquiel sahaquiel 849 Feb 20 15:00 input.log
-rw-r--r-- 1 sahaquiel sahaquiel  14 Feb 20 15:00 logsize.output