Diff between two remote folders through SSH

Question

Does anyone know how to list the files that exists in one remote folder and not in another remote folder. I have two servers (say Server1 and Server2) with similar folder structure where I'm doing Rsync. However, the destination folder has more files than the source as some of the files were deleted. Now I'm trying to find a way to find which files are new in Server2 by using diff between Server 1 and Server 2.

I can take the diff between two local folders directly using the following command:

diff /home/www/images/test_images /var/www/site/images/test_images

But I was wondering if it is possible to diff folders between two remote servers using ssh. Like this?

diff [email protected]:/home/www/images/test_images [email protected]:/var/www/site/images/test_images

Say the ssh configurations of Server 1 and Server 2 are as follows:

Server 1

IP: images.server1.com
User: ubuntu1
Password: pa$$word1
Images Path: /home/www/images/test_images

Server 2

IP: images.server2.com
User: ubuntu2
Password: pa$$word2
Images Path: /var/www/site/images/test_images

Hoping for any help to solve this problem. Thanks.

Answer 1

Try this command:

diff -B <(sshpass -p 'pa$$word1' ssh [email protected] "find /home/www/images/test_images -type f | sed 's/\/home\/www\/images\/test_images\///g'" | sort -n) <(sshpass -p 'pa$$word2' ssh [email protected] "find /var/www/site/images/test_images -type f | sed 's/\/var\/www\/site\/images\/test_images\///g'" | sort -n) | grep ">" | awk '{print $2}'

Explanation:

You can use diff -B <() <() for taking the diff between two streams. The command first uses sshpass to ssh into the two servers without having to enter your passwords interactively.

Each parameter for diff -B uses find command to recursively list all your images in the specified directory and uses sed to remove the root path of the files (because they are different for two servers - and to make it work for the diff command); and the sort command to sort them.

Since the output of the diff command returns either > or <, grep is used to filter out only the diffs from your Server 2. Last, awk prints out only the second column (removes the > column from the output).

NOTE: You need to install sshpass first. Use apt-get to install it as follows:

sudo apt-get install sshpass

You can extend this by piping other commands like rm. Hope this works for you.