Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to compare two huge text files (more than 50gb each)?

I have 2 huge text files and want to find the difference between them? What is the fast command/ utility/ or script to do this job?

I try using unix diff but, it failed for huge files. It show me "Permission Denied"

Also, I try unix bdiff (as I read that is good for huge files) but, I did not complete test it because it show me "bdiff: command not found"

Also, I try windows FC (file compare) but, I did not know how to show the output in a new text file. The result output in the cmd and I can't figure it all.

Any suggestion will help me a lot. Please help me in this matter.

Thanks a lot.

like image 874
sara Avatar asked Jan 07 '23 06:01

sara


2 Answers

You can try diff with the --speed-large-files option :

diff --speed-large-files file1 file2

cmp is another alternative (compare files byte by byte) :

cmp file1 file2
like image 118
SLePort Avatar answered Jan 08 '23 19:01

SLePort


If fc works for you, you can send the output of fc to text file with:

fc file1 file2 >output.txt
like image 33
Munir Avatar answered Jan 08 '23 20:01

Munir