Bash script to compare files

Question

I have a folder with a ton of old photos with many duplicates. Sorting it by hand would take ages, so I wanted to use the opportunity to use bash.

Right now I have the code:

#!/bin/bash

directory="~/Desktop/Test/*"
for file in ${directory};
do
    for filex in ${directory}:
    do
        if [ $( diff {$file} {$filex} ) == 0 ]
        then
            mv ${filex} ~/Desktop
            break
        fi
    done
done 

And getting the exit code:

diff: {~/Desktop/Test/*}: No such file or directory
diff: {~/Desktop/Test/*:}: No such file or directory
File_compare: line 8: [: ==: unary operator expected

I've tried modifying working code I've found online, but it always seems to spit out some error like this. I'm guessing it's a problem with the nested for loop?

Also, why does it seem there are different ways to call variables? I've seen examples that use ${file}, "$file", and "${file}".

Answer 1

In addition to the problems Jonathan Leffler pointed out:

directory="~/Desktop/Test/*"
for file in ${directory};

~ and * won't get expanded inside double-quotes; the * will get expanded when you use the variable without quotes, but since the ~ won't, it's looking for files under an directory actually named "~" (not your home directory), it won't find any matches. Also, as Jonathan pointed out, using variables (like ${directory}) without double-quotes will run you into trouble with filenames that contain spaces or some other metacharacters. The better way to do this is to not put the wildcard in the variable, use it when you reference the variable, with the variable in double-quotes and the * outside them:

directory=~/"Desktop/Test"
for file in "${directory}"/*;

Oh, and another note: when using mv in a script it's a good idea to use mv -i to avoid accidentally overwriting another file with the same name.

And: use shellcheck.net to sanity-check your code and point out common mistakes.

Answer 2

You have the {} in the wrong places:

if [ $( diff {$file} {$filex} ) == 0 ]

They should be at:

if [ $( diff ${file} ${filex} ) == 0 ]

(though the braces are optional now), but you should allow for spaces in the file names:

if [ $( diff "${file}" "${filex}" ) == 0 ]

Now it simply doesn't work properly because when diff finds no differences, it generates no output (and you get errors because the == operator doesn't expect nothing on its left-side). You could sort of fix it by double quoting the value from $(…) (if [ "$( diff … )" == "" ]), but you should simply and directly test the exit status of diff:

if diff "${file}" "${filex}"
then : no difference
else : there is a difference
fi

and maybe for comparing images you should be using cmp (in silent mode) rather than diff:

if cmp -s "$file" "$filex"
then : no difference
else : there is a difference
fi