Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Find shortest matches between two strings

Tags:

python

regex

I have a large log file, and I want to extract a multi-line string between two strings: start and end.

The following is sample from the inputfile:

start spam
start rubbish
start wait for it...
    profit!
here end
start garbage
start second match
win. end

The desired solution should print:

start wait for it...
    profit!
here end
start second match
win. end

I tried a simple regex but it returned everything from start spam. How should this be done?

Edit: Additional info on real-life computational complexity:

  • actual file size: 2GB
  • occurrences of 'start': ~ 12 M, evenly distributed
  • occurences of 'end': ~800, near the end of the file.
like image 845
Eero Aaltonen Avatar asked Dec 07 '22 01:12

Eero Aaltonen


1 Answers

This regex should match what you want:

(start((?!start).)*?end)

Use re.findall method and single-line modifier re.S to get all the occurences in a multi-line string:

re.findall('(start((?!start).)*?end)', text, re.S)

See a test here.

like image 64
famousgarkin Avatar answered Dec 08 '22 15:12

famousgarkin