How to use compile_commands.json with clang python bindings?

Question

I have the following script that attempts to print out all the AST nodes in a given C++ file. This works fine when using it on a simple file with trivial includes (header file in the same directory, etc).

#!/usr/bin/env python
from argparse import ArgumentParser, FileType
from clang import cindex


def node_info(node):
    return {'kind': node.kind,
            'usr': node.get_usr(),
            'spelling': node.spelling,
            'location': node.location,
            'file': node.location.file.name,
            'extent.start': node.extent.start,
            'extent.end': node.extent.end,
            'is_definition': node.is_definition()
            }


def get_nodes_in_file(node, filename, ls=None):
    ls = ls if ls is not None else []
    for n in node.get_children():
        if n.location.file is not None and n.location.file.name == filename:
            ls.append(n)
            get_nodes_in_file(n, filename, ls)
    return ls


def main():
    arg_parser = ArgumentParser()
    arg_parser.add_argument('source_file', type=FileType('r+'),
                            help='C++ source file to parse.')
    arg_parser.add_argument('compilation_database', type=FileType('r+'),
                            help='The compile_commands.json to use to parse the source file.')
    args = arg_parser.parse_args()
    compilation_database_path = args.compilation_database.name
    source_file_path = args.source_file.name
    clang_args = ['-x', 'c++', '-std=c++11', '-p', compilation_database_path]
    index = cindex.Index.create()
    translation_unit = index.parse(source_file_path, clang_args)
    file_nodes = get_nodes_in_file(translation_unit.cursor, source_file_path)
    print [p.spelling for p in file_nodes]


if __name__ == '__main__':
    main()

However, I get a clang.cindex.TranslationUnitLoadError: Error parsing translation unit. when I run the script and provide a valid C++ file that has a compile_commands.json file in its parent directory. This code runs and builds fine using CMake with clang, but I can't seem to figure out how to pass the argument for pointing to the compile_commands.json correctly.

I also had difficulty finding this option in the clang documentation and could not get -ast-dump to work. However, clang-check works fine by just passing the file path!

Answer 1

Your own accepted answer is incorrect. libclang does support compilation databases and so does cindex.py, the libclang python binding.

The main source of confusion might be that the compilation flags that libclang knows/uses are only a subset of all arguments that can be passed to the clang frontend. The compilation database is supported but does not work automatically: it must be loaded and queried manually. Something like this should work:

#!/usr/bin/env python
from argparse import ArgumentParser, FileType
from clang import cindex

compilation_database_path = args.compilation_database.name
source_file_path = args.source_file.name
index = cindex.Index.create()

# Step 1: load the compilation database
compdb = cindex.CompilationDatabase.fromDirectory(compilation_database_path)

# Step 2: query compilation flags
try:
    file_args = compdb.getCompileCommands(source_file_path)
    translation_unit = index.parse(source_file_path, file_args)
    file_nodes = get_nodes_in_file(translation_unit.cursor, source_file_path)
    print [p.spelling for p in file_nodes]
except CompilationDatabaseError:
    print 'Could not load compilation flags for', source_file_path