Using Rust returned array in Python using ctypes

Question

I have a Rust function that returns an array and I want to use this array in Python, it could be a list or numpy.array it does not really matter.

My Rust function looks like this:

#[no_mangle]
pub extern fn make_array() -> [i32; 4] {
    let my_array: [i32; 4] = [1,2,3,4];
    return my_array;
}

And I am trying to call it in Python like this:

In [20]: import ctypes

In [21]: from ctypes import cdll

In [22]: lib = cdll.LoadLibrary("/home/user/RustStuff/embed/target/release/libembed.so")

In [23]: lib.make_array.restype = ctypes.ARRAY(ctypes.c_int32, 4)

In [24]: temp = lib.make_array()

In [25]: [i for i in temp]
Out[25]: [1, 2, -760202930, 32611]

What am I doing wrong? Why is my output not [1,2,3,4]? Why are my first two elements right and other two are populated with garbage?

I was not able to find any good documentation on ctypes.ARRAY, so I just went with what looked right, so that is likely the problem.

Answer 1

As others have said, you can't really return a fixed-size array properly. But you can trick ctypes into doing the right thing by wrapping the array in a structure:

import ctypes

class Int32_4(ctypes.Structure):
    _fields_ = [("array", ctypes.c_int32 * 4)]

lib = ctypes.CDLL("embed.dll")
lib.make_array.restype = Int32_4

temp = lib.make_array()

print(temp.array[:])

This results in [1, 2, 3, 4] on my machine.

Addendum: This is a "trick" because we're exploiting a difference between what C can do and what Rust can do. C won't let you return a fixed-sized array by value, but Rust will, and it works the same as returning a user-defined structure.

So, we do something that C will allow: returning a structure which happens to contain a fixed-size array. This, it is fine with, and it matches the layout that Rust is using.

Of course, this is also somewhat hacky, in that I'm not entirely convinced that this is well-defined behaviour. If you want to be extra safe, you could change the return type on the Rust side to match C:

#[repr(C)]
struct Int32_4 {
    array: [i32; 4]
}

Answer 2

I agree with what @delnan said - you can't return fixed-size arrays in C. One main incompatibility is that Rust arrays know what size they are, but C arrays do not. You will need to abide by how every other C program has done this - return a pointer and a length separately. Isn't Rust a nice modern language in comparison?

I stole and modified some Python code from another answer

import ctypes

from ctypes import cdll

lib = cdll.LoadLibrary("libarray.dylib")
lib.make_array.restype = ctypes.POINTER(ctypes.c_int32 * 4)

print [i for i in lib.make_array().contents]

This works with this Rust code:

static ARRAY: [i32; 4] = [1,2,3,4];

#[no_mangle]
pub extern fn make_array() -> *const i32 {
    ARRAY.as_ptr()
}

Here, we are doing the simplest thing, creating an array that will live for the entire length of the program and returning a reference to its data. In your real program, you will likely need to take more care to ensure that your Vec<i32> or &[i32] strictly outlives how long the Python code has the pointer, else you will cause memory corruption.