grep -r in python

Question

i'd like to implement the unix command 'grep -r' in a python function. i know about commands.getstatusoutput(), but for now i don't want to use that. i came up with this:

def grep_r (str, dir):
    files = [ o[0]+"/"+f for o in os.walk(dir) for f in o[2] if os.path.isfile(o[0]+"/"+f) ]
    return [ l for f in files for l in open(f) if str in l ]

but that of course doesn't use a regex, it just checks if 'str' is a substring of 'l'. so i tried the following:

def grep_r (pattern, dir):
    r = re.compile(pattern)
    files = [ o[0]+"/"+f for o in os.walk(dir) for f in o[2] if os.path.isfile(o[0]+"/"+f) ]
    return [ l for f in files for l in open(f) if r.match(l) ]

but that doesn't work, it doesn't give me any matches even where the former function did. what changed? i could just split it up into a bunch of nested loops, but i'm more interested in being succinct than readable.

Answer 1

You might want to search() instead of match() to catch matches in the middle of lines, as noted in http://docs.python.org/library/re.html#matching-vs-searching

Also, the structure and intent of your code is quite hidden. I've pythonized it.

def grep_r (pattern, dir):
    r = re.compile(pattern)
    for parent, dnames, fnames in os.walk(dir):
        for fname in fnames:
            filename = os.path.join(parent, fname)
            if os.path.isfile(filename):
                with open(filename) as f:
                    for line in f:
                        if r.search(line):
                            yield line

Answer 2

re.match only checks the beginning of the string.

Use re.search()

From the docs:

Python offers two different primitive operations based on regular expressions: match checks for a match only at the beginning of the string, while search checks for a match anywhere in the string (this is what Perl does by default).