Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python is passing 32bit pointer address to C functions

I would like to call my C functions within a shared library from Python scripts. Problem arrises when passing pointers, the 64bit addresses seem to be truncated to 32bit addresses within the called function. Both Python and my library are 64bit.

The example codes below demonstrate the problem. The Python script prints the address of the data being passed to the C function. Then, the address received is printed from within the called C function. Additionally, the C function proves that it is 64bit by printing the size and address of locally creating memory. If the pointer is used in any other way, the result is a segfault.

CMakeLists.txt

cmake_minimum_required (VERSION 2.6) 
add_library(plate MODULE plate.c)

plate.c

#include <stdio.h>
#include <stdlib.h>

void plate(float *in, float *out, int cnt)
{
    void *ptr = malloc(1024);
    fprintf(stderr, "passed address: %p\n", in);
    fprintf(stderr, "local pointer size: %lu\n local pointer address: %p\n", sizeof(void *), ptr);
    free(ptr);
}

test_plate.py

import numpy
import scipy
import ctypes

N = 3
x = numpy.ones(N, dtype=numpy.float32)
y = numpy.ones(N, dtype=numpy.float32)
plate = ctypes.cdll.LoadLibrary('libplate.so')

print 'passing address: %0x' % x.ctypes.data
plate.plate(x.ctypes.data, y.ctypes.data, ctypes.c_int(N))

Output from python-2.7

In [1]: run ../test_plate.py

passing address: 7f9a09b02320

passed address: 0x9b02320

local pointer size: 8

local pointer address: 0x7f9a0949a400

like image 676
papahabla Avatar asked May 19 '13 13:05

papahabla


People also ask

Is ctypes part of Python?

ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries. It can be used to wrap these libraries in pure Python.

What is CDLL in Python?

It is a big C++ codebase with a (very) thin python wrapper which uses CDLL to load the C++ and call some C functions that are available to allow primitive python scripting of the code.

What is C_char_p?

c_char_p is a subclass of _SimpleCData , with _type_ == 'z' . The __init__ method calls the type's setfunc , which for simple type 'z' is z_set . In Python 2, the z_set function (2.7.

What is C_ubyte?

The cubit, generally taken as equal to 18 inches (457 mm), was based on the length of the arm from the elbow to the tip of the middle finger and was considered the equivalent of 6 palms or 2 spans.


1 Answers

The problem is that the ctypes module doesn't check the function signature of the function you're trying to call. Instead, it bases the C types on the Python types, so the line...

plate.plate(x.ctypes.data, y.ctypes.data, ctypes.c_int(N))

...is passing the the first two params as integers. See eryksun's answer for the reason why they're being truncated to 32 bits.

To avoid the truncation, you'll need to tell ctypes that those params are actually pointers with something like...

plate.plate(ctypes.c_void_p(x.ctypes.data),
            ctypes.c_void_p(y.ctypes.data),
            ctypes.c_int(N))

...although what they're actually pointers to is another matter - they may not be pointers to float as your C code assumes.


Update

eryksun has since posted a much more complete answer for the numpy-specific example in this question, but I'll leave this here, since it might be useful in the general case of pointer truncation for programmers using something other than numpy.

like image 81
Aya Avatar answered Nov 10 '22 05:11

Aya