How to preprocess a C source code for pycparser

Question

I need to parse my C and C++ codes with pycparser, but it needs to be stripped of preprocessor directives and comments before.

So, do you know any way to do this? I've found CPP preprocessor, but I have no idea, if I can use it just like that, without "full" preprocessing.

I've found a tool unifdef too, which seems to do exactly what I want, but only with preprocessor conditionals (e.g. #ifdef).

I don't want to write this tool by myself, because it's going to be used with a pretty large project, so I want to use something really sophisticated.

---

My attempts:

I've tried to find places, where was the function test called in this code:

#include <stdio.h>

// asdfdsa
/* sadfsd
 * 
 */

void test() {
    printf("asd");
}

int main() {

    test();

    test();

    return 0;
}

I preprocessed this code with command gcc -E -std=c99 test.c -o testP.c and then I tried to find function calls with this Python code:

#-----------------------------------------------------------------
# pycparser: func_defs.py
#
# Using pycparser for printing out all the calls of some function
# in a C file.
#
# Copyright (C) 2008-2015, Eli Bendersky
# License: BSD
#-----------------------------------------------------------------
from __future__ import print_function
import sys

# This is not required if you've installed pycparser into
# your site-packages/ with setup.py
sys.path.extend(['.', '..'])

from pycparser import c_parser, c_ast, parse_file


# A visitor with some state information (the funcname it's
# looking for)
#
class FuncCallVisitor(c_ast.NodeVisitor):
    def __init__(self, funcname):
        self.funcname = funcname

    def visit_FuncCall(self, node):
        if node.name.name == self.funcname:
            print('%s called at %s' % (self.funcname, node.name.coord))


def show_func_calls(filename, funcname):
    ast = parse_file(filename, use_cpp=True)
    v = FuncCallVisitor(funcname)
    v.visit(ast)

if __name__ == "__main__":
    if len(sys.argv) > 2:
        filename = sys.argv[1]
        func = sys.argv[2]
    else:
        filename = 'test.c'
        func = 'test'

    show_func_calls(filename, func)

but still, I'm getting this error:

Traceback (most recent call last):
  File "func_calls.py", line 46, in <module>
    show_func_calls(filename, func)
  File "func_calls.py", line 33, in show_func_calls
    ast = parse_file(filename, use_cpp=True)
  File "/usr/local/lib/python2.7/dist-packages/pycparser/__init__.py", line 93, in parse_file
    return parser.parse(text, filename)
  File "/usr/local/lib/python2.7/dist-packages/pycparser/c_parser.py", line 146, in parse
    debug=debuglevel)
  File "/usr/local/lib/python2.7/dist-packages/pycparser/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/usr/local/lib/python2.7/dist-packages/pycparser/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/usr/local/lib/python2.7/dist-packages/pycparser/c_parser.py", line 1691, in p_error
    column=self.clex.find_tok_column(p)))
  File "/usr/local/lib/python2.7/dist-packages/pycparser/plyparser.py", line 55, in _parse_error
    raise ParseError("%s: %s" % (coord, msg))
pycparser.plyparser.ParseError: /usr/lib/gcc/x86_64-linux-gnu/4.9/include/stdarg.h:40:27: before: __gnuc_va_list

Answer 1

pycparser documentation specifically says you should use cpp or gcc -E to prepare source code for parsing. So the "full" preprocessing is not an issue of cpp, it's the feature you need in order to run pycparser on your code.

If you simply strip your code of all preprocessor directives, parsing will fail because of undeclared types (like int32_t or size_t) and library functions missing prototypes.

EDIT: pycparser only supports C99 syntax. If gcc takes your code for C++ for some reason, run the preprocessor like this:

gcc -E -std=c99

EDIT2: it seems that you're getting more errors related to compiler-specific symbols. Try to use the "fake" headers provided by pycparser:

gcc -E -std=c99 -I/path/to/pycparser/utils/fake_libc_include