Find the files existing in one directory but not in the other [closed]

Question

diff -r dir1 dir2 | grep dir1 | awk '{print $4}' > difference1.txt

Explanation:

• diff -r dir1 dir2 shows which files are only in dir1 and those only in dir2 and also the changes of the files present in both directories if any.

• diff -r dir1 dir2 | grep dir1 shows which files are only in dir1

• awk to print only filename.

This should do the job:

diff -rq dir1 dir2

Options explained (via diff(1) man page):

• -r - Recursively compare any subdirectories found.
• -q - Output only whether files differ.

comm -23 <(ls dir1 |sort) <(ls dir2|sort)

This command will give you files those are in dir1 and not in dir2.

About <( ) sign, you can google it as 'process substitution'.

A good way to do this comparison is to use find with md5sum, then a diff.

Example:

Use find to list all the files in the directory then calculate the md5 hash for each file and pipe it to a file:

find /dir1/ -type f -exec md5sum {} \; > dir1.txt

Do the same procedure to the another directory:

find /dir2/ -type f -exec md5sum {} \; > dir2.txt

Then compare the result two files with "diff":

diff dir1.txt dir2.txt

This strategy is very useful when the two directories to be compared are not in the same machine and you need to make sure that the files are equal in both directories.

Another good way to do the job is using git

git diff --no-index dir1/ dir2/

Best regards!

Meld (http://meldmerge.org/) does a great job at comparing directories and the files within.

vim's DirDiff plugin is another very useful tool for comparing directories.

vim -c "DirDiff dir1 dir2"

It not only lists which files are different between the directories, but also allows you to inspect/modify with vimdiff the files that are different.

Unsatisfied with all the replies, since most of them work very slowly and produce unnecessarily long output for large directories, I wrote my own Python script to compare two folders.

Unlike many other solutions, it doesn't compare contents of the files. Also it doesn't go inside subdirectories which are missing in another directory. So the output is quite concise and the script works fast.

#!/usr/bin/env python3

import os, sys

def compare_dirs(d1: "old directory name", d2: "new directory name"):
    def print_local(a, msg):
        print('DIR ' if a[2] else 'FILE', a[1], msg)
    # ensure validity
    for d in [d1,d2]:
        if not os.path.isdir(d):
            raise ValueError("not a directory: " + d)
    # get relative path
    l1 = [(x,os.path.join(d1,x)) for x in os.listdir(d1)]
    l2 = [(x,os.path.join(d2,x)) for x in os.listdir(d2)]
    # determine type: directory or file?
    l1 = sorted([(x,y,os.path.isdir(y)) for x,y in l1])
    l2 = sorted([(x,y,os.path.isdir(y)) for x,y in l2])
    i1 = i2 = 0
    common_dirs = []
    while i1<len(l1) and i2<len(l2):
        if l1[i1][0] == l2[i2][0]:      # same name
            if l1[i1][2] == l2[i2][2]:  # same type
                if l1[i1][2]:           # remember this folder for recursion
                    common_dirs.append((l1[i1][1], l2[i2][1]))
            else:
                print_local(l1[i1],'type changed')
            i1 += 1
            i2 += 1
        elif l1[i1][0]<l2[i2][0]:
            print_local(l1[i1],'removed')
            i1 += 1
        elif l1[i1][0]>l2[i2][0]:
            print_local(l2[i2],'added')
            i2 += 1
    while i1<len(l1):
        print_local(l1[i1],'removed')
        i1 += 1
    while i2<len(l2):
        print_local(l2[i2],'added')
        i2 += 1
    # compare subfolders recursively
    for sd1,sd2 in common_dirs:
        compare_dirs(sd1, sd2)

if __name__=="__main__":
    compare_dirs(sys.argv[1], sys.argv[2])

Sample usage:

user@laptop:~$ python3 compare_dirs.py dir1/ dir2/
DIR  dir1/out/flavor-domino removed
DIR  dir2/out/flavor-maxim2 added
DIR  dir1/target/vendor/flavor-domino removed
DIR  dir2/target/vendor/flavor-maxim2 added
FILE dir1/tmp/.kconfig-flavor_domino removed
FILE dir2/tmp/.kconfig-flavor_maxim2 added
DIR  dir2/tools/tools/LiveSuit_For_Linux64 added

Or if you want to see only files from the first directory:

user@laptop:~$ python3 compare_dirs.py dir2/ dir1/ | grep dir1
DIR  dir1/out/flavor-domino added
DIR  dir1/target/vendor/flavor-domino added
FILE dir1/tmp/.kconfig-flavor_domino added

P.S. If you need to compare file sizes and file hashes for potential changes, I published an updated script here: https://gist.github.com/amakukha/f489cbde2afd32817f8e866cf4abe779