Using grep recursively

Question

grep has the ability to search recursively using the -r option. However, I was wondering if grep has the ability to search for a query string recursively for a specified number of subfolder levels. For instance, I have a folder root which has folders parent1, parent2, ..., parentN. Each parent folder has normal text files and folders called child1, child2, ..., childM. I'd like to run grep from the root level and search in the files inside the parents without looking in the child folders. Is there a simple way to do this?

Answer 1

As Kent notes, you can't do this with a straight grep; it simply isn't powerful enough. The trick is to use find to work out which files to search, and pass the file list that find produces to grep.

If you run man find, you'll get a man page of the many options that find takes. The one we're interested in here is -maxdepth.

Let's build up the command we need. Run the commands at each stage to see what it looks like:

• find . will list all the files and folders that exist in the current folder (.) or any descendant folder.

• find . -maxdepth 1 will list all the files and folders in the current folder. find . -maxdepth 2 will likewise list for all files and folders in the current folder and any immediate subfolders. And so forth…

• Note that we're getting folders listed too; we don't want this, as grep can't search a folder itself, only the files in the folder. Add -type f to only get files listed: find . -maxdepth 2 -type f.

Now we know the files we want to search, we need to get grep to search those files. The standard way to do this is using xargs:

find . -maxdepth 2 -type f | xargs grep <text-to-search-for>

The | takes "standard output", aka "stdout", from find (ie what you normally see on screen) and pipes it into xarg's "standard input", aka "stdin", ie what would normally happen if you were typing into a running program.

xargs is a cunning little program that runs whatever you tell it to (here, grep <text-to-search-for>), after adding all the arguments it received on stdin. The result is that grep will search every file that find finds.

This doesn't work if some of your file names have spaces in them, though, because xargs thinks the space is separating two different file names, rather than a part of one file name. There are a bunch of ways to handle this (the ideal is to just not use spaces in file names), and the common one is to use one of the fancier features of find.

If you add an -exec argument to find, it will execute everything you specify up to a ; or +. If you add a {} (that's the literal characters { and }), it will replace that with a list of all the files. Since find is doing that, it knows that spaces in the filename are supposed to be in the filename.

Therefore the best way to do what you're trying to is thus:

find . -type f -maxdepth 2 -exec grep <text-to-search-for> {} +

(The difference between ending with + and ; makes no difference here. If you're interested it's in man find, but the short version is that + is faster but means you can only have one {} in the command.)

Answer 2

you can try these out:

grep:

 --exclude=GLOB
              Skip files whose base name matches GLOB  (using
              wildcard  matching).   A file-name  glob  can  use *,
              ?, and [...]  as wildcards, and \ to quote a wildcard
              or backslash character literally.

       --exclude-from=FILE
              Skip files whose base name matches any of the file-name
              globs  read  from FILE (using wildcard matching as
              described under --exclude).

       --exclude-dir=DIR
              Exclude directories matching the pattern DIR from
              recursive searches.

or use this find | xargs grep

with find, you could define the level

EDIT

pipe output of one command to another is very common in linux/unix world. I bet you do it everyday.

echo "abc"|sed 's/a/x/'
find . -name "*.pyc" |xargs rm
awk 'blahblah' file | sort |head -n2 
tree|grep 'foo'
mvn compile|ack 'error'
...

Note that not all examples above are efficient. They are just examples.