Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Fastest way to grep through thousands of gz files?

I have thousands of .gz files all in one directory. I need to grep through them for the string Mouse::Handler, is the following the fastest (and most accurate) way to do this?

find . -name "*.gz" -exec zgrep -H 'Mouse::Handler' {} \;

Ideally I would also like to print out the line that I find this string on.

I'm running on a RHEL linux box.

like image 894
Nosrettap Avatar asked Aug 15 '14 00:08

Nosrettap


People also ask

Does grep work on GZ files?

Sometimes you need to search the contents of . gz files in your system. Unfortunately, grep doesn't work on compressed files. To overcome this, people usually advise to first uncompress the file(s), and then grep your text, after that finally re-compress your file(s)…

How do I grep a GZ file in Linux?

Solution: the zgrep command The Linux zgrep command works just like the grep command, except it works on text files that have been compressed with the gzip command. As you can see, using the zgrep command is much easier than using the three-step gunzip/grep gzip command I showed at the beginning of this article.

Can you use awk on GZ file?

Awk doesn't read the . gz file. It still doesn't work.


1 Answers

You can search in parallel using

find . -name "*.gz" | xargs -n 1 -P NUM  zgrep -H 'Mouse::Handler' 

where NUM is around the number of cores you have.

like image 122
Jordan Samuels Avatar answered Sep 28 '22 07:09

Jordan Samuels