Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to find a pattern and surrounding content in a very large SINGLE line file?

Tags:

bash

parsing

I have a very large file 100Mb+ where all the content is on one line. I wish to find a pattern in that file and a number of characters around that pattern.

For example I would like to call a command like the one below but where -A and -B are number of bytes not lines:

cat very_large_file | grep -A 100 -B 100 somepattern

So for a file containing content like this:

1234567890abcdefghijklmnopqrstuvwxyz

With a pattern of

890abc
and a before size of -B 3 
and an after size of -A 3

I want it to return:

567890abcdef

Any tips would be great. Many thanks.

like image 711
emson Avatar asked Oct 03 '11 18:10

emson


1 Answers

You could try the -o option:

-o, --only-matching
      Show only the part of a matching line that matches PATTERN.

and use a regular expression to match your pattern and the 3 preceding/following characters i.e.

grep -o -P ".{3}pattern.{3}" very_large_file 

In the example you gave, it would be

echo "1234567890abcdefghijklmnopqrstuvwxyz" > tmp.txt
grep -o -P ".{3}890abc.{3}" tmp.txt
like image 110
Paolo Tedesco Avatar answered Oct 11 '22 14:10

Paolo Tedesco