Function to replace duplicate files with hardlinks

Question

I need to write a bash script that iterates through the files of a specified directory and replaces duplicates of files with hardlinks.

So far, I have iterated through the files and stored the file names in an array.

Now, I need to go through that array and check each file for any duplicates. The relevant code that I'm using to do this is below:

#"files" is the array with all the file names
#"fileNum" is the size of the array

...

for((j=0; j<$fileNum; j++))             #for every file
do
    if [ -f "$files[$j]" ]          #access that file in the array
    then
        for((k=0; k<$fileNum; k++))     #for every other file
        do
            if [ -f "$files[$k]" ]      #access other files in the array
            then
                test[cmp -s ${files[$j]} ${files[$k]}]      #compare if the files are identical
                [ln ${files[$j]} ${files[$k]}]          #change second file to a hard link
            fi
...

The test directory has two files: a and b. b is a copy of a.

After running the script, ls -l shows that all of the files still only have 1 hardlink, so the script appears to have basically done nothing.

Where am I going wrong?

Answer 1

First of all, don't iterate over all files twice otherwise you will compare each couple twice and you will compare a file against itself. Secondly, dollar signs are not required inside (( )). Finally, I don't think you access the array correctly, so the [ -f ] always fails and hence nothing happens. Note that this requires a change in your first loop as well (the array notation when using [ -f ].

The second loop, instead, should be like this:

for((k=j+1; k<fileNum; k++)); do
    if [ -f ${files["$k"]} ]; then
        cmp ${files["$j"]} ${files["$k"]}
        if [[ "$?" -eq 0 ]]; then
            rm  ${files["$k"]}
            ln ${files["$j"]} ${files["$k"]}
        fi
    fi
done

If the cmp here succeeds it will create a link. cmp only fails when the files differ. Try this code and let me know when you have any problems.

Alternatively:

for((k=j+1; k<fileNum; k++)); do
    if [ -f ${files["$k"]} ]; then
        cmp ${files["$j"]} ${files["$k"]} && ln -f ${files["$j"]} ${files["$k"]}
    fi
done