Can python load definitions from a C header file?

Question

I'm writing a python Wrapper around a C API. I have an extensive API description and right now I'm struggling with the implementation of enums defined in the header file.

Let's assume i have a C API function inside myAPI.dll, that accepts an enum as argument like:

void SomeFunction(SomeEnum data)

From the header file, I can see that SomeEnum looks like:

enum SomeEnum{
    SomeValue = 1,
    SomeOtherValue = 2,
    SomeVeryStupidValue = -1
};

In python, I load the .dll like:

myAPI = ctypes.cdll.LoadLibrary('myAPI.dll')

now I would like to be able to call:

myAPI.SomeFunction(SomeValue)

I know, that I could define SomeValue in python, but it would be convenient to load its definition directly from the header file or have it directly as an attribute of myAPI. Is this possible?

Answer 1

It's possible. I wrote a tool years ago to scan a file for C++ enum syntax using pyparsing. It's now a pyparsing example that I've reproduced here in case the link changes. As you can see the file doesn't have to even be entirely valid C++. It defines the enum grammar and scans the file for text matching the grammar, generating Python variables.

#
# cpp_enum_parser.py
#
# Posted by Mark Tolonen on comp.lang.python in August, 2009,
# Used with permission.
#
# Parser that scans through C or C++ code for enum definitions, and
# generates corresponding Python constant definitions.
#
#

from pyparsing import *

# sample string with enums and other stuff
sample = """
    stuff before
    enum hello {
        Zero,
        One,
        Two,
        Three,
        Five=5,
        Six,
        Ten=10
        };
    in the middle
    enum blah
        {
        alpha,
        beta,
        gamma = 10 ,
        zeta = 50
        };
    at the end
    """

# syntax we don't want to see in the final parse tree
LBRACE, RBRACE, EQ, COMMA = map(Suppress, "{}=,")
_enum = Suppress("enum")
identifier = Word(alphas, alphanums + "_")
integer = Word(nums)
enumValue = Group(identifier("name") + Optional(EQ + integer("value")))
enumList = Group(enumValue + ZeroOrMore(COMMA + enumValue))
enum = _enum + identifier("enum") + LBRACE + enumList("names") + RBRACE

# find instances of enums ignoring other syntax
for item, start, stop in enum.scanString(sample):
    id = 0
    for entry in item.names:
        if entry.value != "":
            id = int(entry.value)
        print("%s_%s = %d" % (item.enum.upper(), entry.name.upper(), id))
        id += 1

Output:

HELLO_ZERO = 0
HELLO_ONE = 1
HELLO_TWO = 2
HELLO_THREE = 3
HELLO_FIVE = 5
HELLO_SIX = 6
HELLO_TEN = 10
BLAH_ALPHA = 0
BLAH_BETA = 1
BLAH_GAMMA = 10
BLAH_ZETA = 50

Answer 2

Adapting the code from Mark Tolonen to create actual Python Enums:

# import the dependencies
from enum import EnumMeta, IntEnum
from pyparsing import Group, Optional, Suppress, Word, ZeroOrMore
from pyparsing import alphas, alphanums, nums

The first step is to create a new EnumMeta type (note that this is one of the times when subclassing EnumMeta is a good idea):

CPPEnum = None
class CPPEnumType(EnumMeta):
    #
    @classmethod
    def __prepare__(metacls, clsname, bases, **kwds):
        # return a standard dictionary for the initial processing
        return {}
    #
    def __init__(clsname, *args , **kwds):
        super(CPPEnumType, clsname).__init__(*args)
    #
    def __new__(metacls, clsname, bases, clsdict, **kwds):
        if CPPEnum is None:
            # first time through, ignore the rest
            enum_dict = super(CPPEnumType, metacls).__prepare__(
                    clsname, bases, **kwds
                    )
            enum_dict.update(clsdict)
            return super(CPPEnumType, metacls).__new__(
                    metacls, clsname, bases, enum_dict, **kwds,
                    )
        members = []
        #
        # remove _file and _name using `pop()` as they will
        # cause problems in EnumMeta
        try:
            file = clsdict.pop('_file')
        except KeyError:
            raise TypeError('_file not specified')
        cpp_enum_name = clsdict.pop('_name', clsname.lower())
        with open(file) as fh:
            file_contents = fh.read()
        #
        # syntax we don't want to see in the final parse tree
        LBRACE, RBRACE, EQ, COMMA = map(Suppress, "{}=,")
        _enum = Suppress("enum")
        identifier = Word(alphas, alphanums + "_")
        integer = Word(nums)
        enumValue = Group(identifier("name") + Optional(EQ + integer("value")))
        enumList = Group(enumValue + ZeroOrMore(COMMA + enumValue))
        enum = _enum + identifier("enum") + LBRACE + enumList("names") + RBRACE
        #
        # find the cpp_enum_name ignoring other syntax and other enums
        for item, start, stop in enum.scanString(file_contents):
            if item.enum != cpp_enum_name:
                continue
            id = 0
            for entry in item.names:
                if entry.value != "":
                    id = int(entry.value)
                members.append((entry.name.upper(), id))
                id += 1
        #
        # get the real EnumDict
        enum_dict = super(CPPEnumType, metacls).__prepare__(clsname, bases, **kwds)
        # transfer the original dict content, names starting with '_' first
        items = list(clsdict.items())
        items.sort(key=lambda p: (0 if p[0][0] == '_' else 1, p))
        for name, value in items:
            enum_dict[name] = value
        # add the members
        for name, value in members:
            enum_dict[name] = value
        return super(CPPEnumType, metacls).__new__(
                metacls, clsname, bases, enum_dict, **kwds,
                )

Once the new type is created, we can create the new base class:

class CPPEnum(IntEnum, metaclass=CPPEnumType):
    pass

Once you have the new CPPEnum base class, using it is as simple as:

class Hello(CPPEnum):
    _file = 'some_header.h'

class Blah(CPPEnum):
    _file = 'some_header.h'
    _name = 'blah'              # in case the name in the file is not the lower-cased
                                # version of the Enum class name (so not needed in
                                # in this case)

And in use:

>>> list(Hello)
[
    <Hello.ZERO: 0>, <Hello.ONE: 1>, <Hello.TWO: 2>, <Hello.THREE: 3>,
    <Hello.FIVE: 5>, <Hello.SIX: 6>, <Hello.TEN: 10>,
    ]

---

Disclosure: I am the author of the Python stdlib Enum, the enum34 backport, and the Advanced Enumeration (aenum) library.