Import constants from .h file into python

Question

I've been looking for a simple answer to this question, but it seems that I can't find one. I would prefer to stay away from any external libraries that aren't already included in Python 2.6/2.7.

I have 2 c header files that resemble the following:

//constants_a.h
const double constant1 = 2.25;
const double constant2 = -0.173;
const int constant3 = 13;

...

//constants_b.h
const double constant1 = 123.25;
const double constant2 = -0.12373;
const int constant3 = 14;

...

And I have a python class that I want to import these constants into:

#pythonclass.py
class MyObject(object):
    def __init(self, mode):
        if mode is "a":
            # import from constants_a.h, like:
            # self.constant1 = constant1
            # self.constant2 = constant2
        elif mode is "b":
            # import from constants_b.h, like:
            # self.constant1 = constant1
            # self.constant2 = constant2

...

I have c code which uses the constants as well, and resembles this:

//computations.c
#include <stdio.h>
#include <math.h>
#include "constants_a.h"

// do some calculations, blah blah blah

How can I import the constants from the header file into the Python class?

The reason for the header files constants_a.h and constants_b.h is that I am using python to do most of the calculations using the constants, but at one point I need to use C to do more optimized calculations. At this point I am using ctypes to wrap the c code into Python. I want to keep the constants away from the code just in case I need to update or change them, and make my code much cleaner as well. I don't know if it helps to note I am also using NumPy, but other than that, no other non-standard Python extensions. I am also open to any suggestions regarding the design or architecture of this program.

Answer 1

In general, defining variables in C header file is poor style. The header file should only declare objects, leaving their definition for the appropriate ".c" source code file.

One thing you may want to do is to declare the library-global constants like extern const whatever_type_t foo; and define (or "implement") them (i.e. assigning values to them) somewhere in your C code (make sure you do this only once).

Anyway, let's ignore how you do it. Just suppose you've already defined the constants and made their symbols visible in your shared object file "libfoo.so". Let us suppose you want to access the symbol pi, defined as extern const double pi = 3.1415926; in libfoo, from your Python code.

Now you typically load your object file in Python using ctypes like this:

>>> import ctypes
>>> libfoo = ctypes.CDLL("path/to/libfoo.so")

But then you'll see ctypes thinks libfoo.pi is a function, not a symbol for constant data!

>>> libfoo.pi
<_FuncPtr object at 0x1c9c6d0>

To access its value, you have to do something rather awkward -- casting what ctypes thinks is a function back to a number.

>>> pi = ctypes.cast(foo.pi, ctypes.POINTER(ctypes.c_double))
>>> pi.contents.value
3.1415926

In C jargon, this vaguely corresponds to the following thing happening: You have a const double pi, but someone forces you to use it only via a function pointer:

typedef int (*view_anything_as_a_function_t)(void);
view_anyting_as_a_function_t pi_view = π

What do you do with the pointer pi_view in order to use the value of pi? You cast it back as a const double * and dereference it: *(const double *)(pi_view).

So this is all very awkward. Maybe I'm missing something but this I believe is by design of the ctypes module -- it's there chiefly for making foreign function calls, not for accessing "foreign" data. And exporting pure data symbol in a loadable library is arguably rare.

And this will not work if the constants are only C macro definitions. There's in general no way you can access macro-defined data externally. They're macro-expanded at compile time, leaving no visible symbol in the generated library file, unless you also export their macro values in your C code.

Answer 2

I recommend using regular expressions (re module) to parse the information you want out of the files.

Building a full C parser would be huge, but if you only use the variables and the file is reasonably simple/predictable/under control, then what you need to write is straightforward.

Just watch out for 'gotcha' artifacts such as commented-out code!

Answer 3

I would recommend using some kind of configuration file readable by both Python and C program, rather than storing constant values in headers. E.g. a simple csv, ini-file, or even your own simple format of 'key:value' pairs. And there will be no need to recompile the C program every time you'd like to change one of the values :)

Answer 4

I'd up-vote emilio, but I'm lacking rep!

Although you have requested to avoid other non-standard libraries, you may wish to take a look at Cython (Cython: C-Extensions for Python www.cython.org/), which offers the flexibility of Python coding and the raw speed of execution of C/C++-compiled code.

This way you can use regular Python for everything, but handle the expensive elements of code using its built-in C-types. You can then convert your Python code into .c files too (or just wrap external C-libraries themselves. ), which can then be compiled into a binary. I've achieved up to 10x speed-ups doing so for numerical routines. I also believe NumPy uses it.