Pycparser failed on comments

Question

When I'm trying to use pycparser to parse files with comments I got ParseError

import pycparser
parser = pycparser.CParser()
parser.parse("int main(void){return 0;}")
parser.parse("/* comment */ int main(void){return 0;}")


Traceback (most recent call last):
File "test_pycparser.py", line 18, in <module> parser.parse("/* comment */ int main(void){return 0;}")
File "build\bdist.win32\egg\pycparser\c_parser.py", line 124, in parse
File "build\bdist.win32\egg\pycparser\ply\yacc.py", line 265, in parse
File "build\bdist.win32\egg\pycparser\ply\yacc.py", line 1047, in parseopt_notrack
File "build\bdist.win32\egg\pycparser\c_parser.py", line 1423, in p_error
File "build\bdist.win32\egg\pycparser\plyparser.py", line 54, in _parse_error
pycparser.plyparser.ParseError: :1:1: before: /

Solution: pycparser in current version doesn't support comments in source code, but this fork allow it, or you can use recipe from question Python snippet to remove C and C++ comments to remove comments from source code.

import pycparser
import re
def comment_remover(text):
    def replacer(match):
        s = match.group(0)
        if s.startswith('/'):
            return ""
        else:
            return s
    pattern = re.compile(
        r'//.*?$|/\*.*?\*/|\'(?:\\.|[^\\\'])*\'|"(?:\\.|[^\\"])*"',
        re.DOTALL | re.MULTILINE
    )
    return re.sub(pattern, replacer, text)

parser = pycparser.CParser(keep_comment=True)
parser.parse("int main(void){return 0;}")
parser.parse("/* comment */ int main(void){return 0;}")
parser_2 = pycparser.CParser()
parser.parse(comment_remover("/* comment */ int main(void){return 0;}"))

Answer 1

Indeed, pycparser does not parse comments, or anything related to the C preprocessor (in a regular C compiler, the preprocessor strips comments before the compiler sees them).

To stop comments from messing your parsing, run the code through the preprocessor first, as suggested in the pycparser README. To actually meaningfully parse comments (and get their contents), pycparser is not the right tool, unfortunately.