Is there a way to load the constant values stored in a header file via ctypes?

Question

I'm doing a bunch of ctypes calls to the underlying OS libraries. My progress slows to a crawl anytime the docs reference a constant value stored in a .h. file somewhere, as I have to go track it down, and figure out what the actual value is so that I can pass it into the function.

Is there any way to load a .h file with ctypes and get access to all of the constants?

Answer 1

No.

Early versions of ctypes came with a module called codegenerator, which would parse header files, both to get the constant values and to convert the prototypes into restype/argtypes declarations. However, as far as I know, this was never finished, and it was dropped from the package before inclusion in the stdlib.

You could dig into the source and pull out the constants stuff while skipping the much more complicated prototypes stuff.

---

However, the way I've usually done it is to write my own generator.

For example, run this script as part of your setup process:

constants = {}
with open('foo.h') as infile:
    for name, value in re.findall(r'#define\s+(\w+)\s+(.*)', infile):
        try:
            constants[name] = ast.literal_eval(value)
        except Exception as e:
            pass # maybe log something
with open('_foo_h.py', w) as outfile:
    outfile.write(repr(constants))

Then foo.py can just from _foo_h import *.

Writing a perfect regexp for this is very hard, maybe impossible; writing one that works for the headers you actually care about in a given project is very easy. In fact, often, either the one above, or one that skips over comments, is all you need.

But sometimes this won't work. For example, a header file might #define FOO_SIZE 8 for 64-bit builds, and #define FOO_SIZE 4 for 32-bit builds. How do you handle that?

For that, you ask the compiler to do it for you. Most compilers have a way to preprocess a file just far enough to get all of the active definitions. Some compilers can even just dump the macro defines, in a nice format, skipping everything else. With gcc and flag-compatible compilers like clang, -E preprocesses, and -dM dumps macros. So:

macros = subprocess.check_output(['gcc', '-dM', '-E', '-', 'foo.h'])
for line in macros.splitlines():
    try:
        _, name, value = line.split(None, 2)
        constants[name] = ast.literal_eval(value)
    except Exception as e:
        pass # again, do something nicer

You may want to pass in some extra compiler flags to control what gets defined appropriately, like the results of pkgconfig foo --cflags.

This will also give you the macros defined in anything that foo.h (recursively) includes, and gcc's builtin macros. You may or may not want each of those. Somewhere among the 69105 gcc flags, I believe there are ways to control that, but I don't remember them.

---

Note that neither of these will get you constant variables or enums, like:

static const int SPAM_SPAM_SPAM = 73;
enum {
    kSPAM = 1,
    kEGGS
};

Parsing that gets more difficult; you'd want to use a real C99 parser like pycparser—or, alternatively, you'd want to parse the output of something like gccxml instead of gcc -E. But even that isn't going to tell you that kEGGS is 2 without you writing a bit of logic.

And if you want to deal with C++, it's even worse, what with constexpr and static class members and user-defined literals…

---

Alternatively… do you have to use ctypes?

CFFI provides a different way to call C code from Python—and it makes this a lot easier.

Cython lets you write almost-Python code that gets compiled to C that gets compiled to a Python extension module, and it can include header files directly.

There are also a variety of binding-generators (e.g., SWIG) or binding-writing libraries (e.g., boost::python) that can make it easier to export values to Python through an extension module.