Bash script to limit a directory size by deleting files accessed last

Question

I had previously used a simple find command to delete tar files not accessed in the last x days (in this example, 3 days):

find /PATH/TO/FILES -type f -name "*.tar" -atime +3 -exec rm {} \;

I now need to improve this script by deleting in order of access date and my bash writing skills are a bit rusty. Here's what I need it to do:

1. check the size of a directory /PATH/TO/FILES
2. if size in 1) is greater than X size, get a list of the files by access date
3. delete files in order until size is less than X

The benefit here is for cache and backup directories, I will only delete what I need to to keep it within a limit, whereas the simplified method might go over size limit if one day is particularly large. I'm guessing I need to use stat and a bash for loop?

Answer 1

I improved brunner314's example and fixed the problems in it.

Here is a working script I'm using:

#!/bin/bash
DELETEDIR="$1"
MAXSIZE="$2" # in MB
if [[ -z "$DELETEDIR" || -z "$MAXSIZE" || "$MAXSIZE" -lt 1 ]]; then
    echo "usage: $0 [directory] [maxsize in megabytes]" >&2
    exit 1
fi
find "$DELETEDIR" -type f -printf "%T@::%p::%s\n" \
| sort -rn \
| awk -v maxbytes="$((1024 * 1024 * $MAXSIZE))" -F "::" '
  BEGIN { curSize=0; }
  { 
  curSize += $3;
  if (curSize > maxbytes) { print $2; }
  }
  ' \
  | tac | awk '{printf "%s\0",$0}' | xargs -0 -r rm
# delete empty directories
find "$DELETEDIR" -mindepth 1 -depth -type d -empty -exec rmdir "{}" \;

Answer 2

Here's a simple, easy to read and understand method I came up with to do this:

DIRSIZE=$(du -s /PATH/TO/FILES | awk '{print $1}')
if [ "$DIRSIZE" -gt "$SOMELIMIT" ]
  then
    for f in `ls -rt --time=atime /PATH/TO/FILES/*.tar`; do
    FILESIZE=`stat -c "%s" $f`
    FILESIZE=$(($FILESIZE/1024))

    DIRSIZE=$(($DIRSIZE - $FILESIZE))
    if [ "$DIRSIZE" -lt "$LIMITSIZE" ]; then
        break
    fi
done
fi

Answer 3

I didn't need to use loops, just some careful application of stat and awk. Details and explanation below, first the code:

find /PATH/TO/FILES -name '*.tar' -type f \
| sed 's/ /\\ /g' \
| xargs stat -f "%a::%z::%N" \
| sort -r \
| awk '
  BEGIN{curSize=0; FS="::"}
  {curSize += $2}
  curSize > $X_SIZE{print $3}
  '
| sed 's/ /\\ /g' \
| xargs rm

Note that this is one logical command line, but for the sake of sanity I split it up.

It starts with a find command based on the one above, without the parts that limit it to files older than 3 days. It pipes that to sed, to escape any spaces in the file names find returns, then uses xargs to run stat on all the results. The -f "%a::%z::%N" tells stat the format to use, with the time of last access in the first field, the size of the file in the second, and the name of the file in the third. I used '::' to separate the fields because it is easier to deal with spaces in the file names that way. Sort then sorts them on the first field, with -r to reverse the ordering.

Now we have a list of all the files we are interested in, in order from latest accessed to earliest accessed. Then the awk script adds up all the sizes as it goes through the list, and begins outputting them when it gets over $X_SIZE. The files that are not output this way will be the ones kept, the other file names go to sed again to escape any spaces and then to xargs, which runs rm them.