python regex to match multi-line preprocessor macro

Question

What follows is a regular expression I have written to match multi-line pre-processor macros in C / C++ code. I'm by no means a regular expressions guru, so I'd welcome any advice on how I can make this better.

Here's the regex:

\s*#define(.*\\\n)+[\S]+(?!\\)

It should match all of this:

#define foo(x) if(x) \
doSomething(x)

But only some of this (shouldn't match the next line of code:

#define foo(x) if(x) \
doSomething(x)
normalCode();

And also shouldn't match single-line preprocessor macros.

I'm pretty sure that the regex above works - but as I said, there probably a better way of doing it, and I imagine that there are ways of breaking it. Can anyone suggest any?

Answer 1

This is a simple test program I knocked up:

#!/usr/bin/env python

TEST1="""
#include "Foo.h"
#define bar foo\\
    x
#include "Bar.h"
"""

TEST2="""
#define bar foo
#define x 1 \\
    12 \\
    2 \\\\ 3
Foobar
"""

TEST3="""
#define foo(x) if(x) \\
doSomething(x)
"""

TEST4="""
#define foo(x) if(x) \\
doSomething(x)
normalCode();
"""

import re
matcher = re.compile(r"^[ \t]*#define(.*\\\n)+.*$",re.MULTILINE)

def extractDefines(s):
    mo = matcher.search(s)
    if not mo:
        print mo
        return
    print mo.group(0)

extractDefines(TEST1)
extractDefines(TEST2)
extractDefines(TEST3)
extractDefines(TEST4)

The re I used:

r"^[ \t]*#define(.*\\\n)+.*$"

Is very similar to the one use used, the changes:

1. [ \t] To avoid newlines at the start of the define.
2. I rely on + being greedy, so I can use a simple .*$ at the end to get the first line of the define that doesn't end with \

Answer 2

start        = r"^\s*#define\s+"
continuation = r"(?:.*\\\n)+"
lastline     = r".*$"

re_multiline_macros = re.compile(start + continuation + lastline, 
                                 re.MULTILINE)