Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to grep for lines which contain particular words in a log file?

I have a big log file which I am trying to scan it for a particular words. In general, I will have few words which I need to grep on my big log file and print out the line which contains those words.

I know how to do simple grep on a file. Suppose if my file name is abc.log and I need to find a line which contains word "hello" then I always do it like this and it prints out the line for me.

grep -i "hello" abc.log

But I don't know how to do the grep for combination of words. Meaning I would have list of words and I will scan my abc.log file for all those words and I will print out the lines which contains those words individually.

#!/bin/bash

data="hello,world,tester"

# find all the lines which contains word hello or world or tester

So in my above shell script I will split my data variable and look for hello word in abc.log so any line which contains hello word, I will print it out and similarly with world and tester as well.

I am trying to make this pretty generic so that I just need to add my list of words in the data variable without touching the actual logic of grepping the logs.

like image 924
john Avatar asked Oct 10 '14 04:10

john


2 Answers

I would use a regular expression, like this:

grep -E 'hello|world|tester' abc.log
like image 58
bruchowski Avatar answered Sep 20 '22 01:09

bruchowski


If you store your patterns in a file, one per line, you can use grep -f file-with-patterns file-to-search.log

From the man page:

   -f FILE, --file=FILE
          Obtain  patterns  from  FILE,  one  per  line.   The  empty file
          contains zero patterns, and therefore matches nothing.   (-f  is
          specified by POSIX.)

Edit 2018:

Since I wrote this, I have become aware of the following interesting edge cases:

  • You can read the list of patterns from pipe using -f - (if you don't need stdin, i.e. you specified files on grep's command line) or -f <() (in any case)
  • grep's performance starts to fail badly if hundreds of patterns are passed. If your use case is that insane, consider generating and immediately executing a sed (or some other language) script, although this could potentially have problems with overlapping patterns.
like image 24
o11c Avatar answered Sep 19 '22 01:09

o11c