Improve Python code performance

Question

How do I improve the performance of this simple piece of python code ? Isn't re.search the best way to find a matching line, since it is almost ~6x slower than Perl or am I doing something wrong ?

#!/usr/bin/env python

import re
import time
import sys

i=0
j=0
time1=time.time()
base_register =r'DramBaseAddress\d+'
for line in  open('rndcfg.cfg'):
    i+=1
    if(re.search(base_register, line)):
        j+=1
time2=time.time()

print (i,j)
print (time2-time1)    
print (sys.version)

This code takes about 0.96 seconds to complete (Average of 10 runs)
Output:

168197 2688
0.8597519397735596
3.3.2 (default, Sep 24 2013, 15:14:17)
[GCC 4.1.1]

while the following Perl code does it in 0.15 seconds.

#!/usr/bin/env perl
use strict;
use warnings;

use Time::HiRes qw(time);

my $i=0;my $j=0;
my $time1=time;
open(my $fp, 'rndcfg.cfg');
while(<$fp>)
{
    $i++;
    if(/DramBaseAddress\d+/)
    {
        $j++;
    }
}
close($fp);
my $time2=time;

printf("%d,%d\n",$i,$j);
printf("%f\n",$time2-$time1);
printf("%s\n",$]);

Output:

168197,2688
0.135579
5.012001

EDIT: Corrected regular expression - Which worsened the performance slightly

Answer 1

actually, regular expression is less efficient than the string methods in Python. From https://docs.python.org/2/howto/regex.html#use-string-methods:

Strings have several methods for performing operations with fixed strings and they’re usually much faster, because the implementation is a single small C loop that’s been optimized for the purpose, instead of the large, more generalized regular expression engine.

replacing re.search with str.find will give you better runtime. otherwise, using the in operator that others suggested would be optimized, too.

as for the speed difference between the Python & Perl version, i'll just chalk it up to the inherent quality of each language: text processing - python vs perl performance

Answer 2

In this case you are using a fixed string, not a regular expression.

For regular strings there are faster methods:

>>> timeit.timeit('re.search(regexp, "banana")', setup = "import re;     regexp=r'nan'")
1.2156920433044434
>>> timeit.timeit('"banana".index("nan")')
0.23752403259277344
>>> timeit.timeit('"banana".find("nan")')
0.2411658763885498

Now this kind of text processing is the sweet spot of Perl (aka Practical Extraction and Reporting Language) (aka Pathological Eclectic Rubbish Lister) and has been optimized extensively over the years. All that collective focus adds up.