Wrap C Function returning array of unknown size using ctypes python

Question

I am trying to wrap a C function using ctypes, which returns a character array of unknown size. The function is from the gdal c api, but my question is not specific to that function.

I would like to know if there is a general way of deconstructing the output of a function returning a char** array object of unknown size. In ctypes, this would be POINTER(c_char_p * X) where X is not known.

Using the tips from an answer to a similar question, I was able to get the following to work:

# Define the function wrapper.
f = ctypes.CDLL('libgdal.so.20').GDALGetMetadata
MAX_OUTPUT_LENGTH = 10
f.restype = ctypes.POINTER(ctypes.c_char_p * MAX_OUTPUT_LENGTH)
f.argtypes = [ctypes.c_void_p, ctypes.c_char_p]

# Example call (the second argument can be null).
result = []
counter = 0
output = f(ptr, None).contents[counter]
while output:
    result.append(output)
    counter += 1
    output = f(ptr, None).contents[counter]

Where output is the resulting array and ptr is a ctypes pointer to an open GDALRaster. The limitation to this is that I have to construct an array with a fixed length before calling the function. I can guess what the maximum length could be in practical cases, and simply use that. But that is arbitrary, and I wonder if there is a way of getting an array pointer without specifying the array's length. In other words:

Is there a way to do something similar as the example above, but without specifying an arbitrary maximum length?

Answer 1

It turns out, that you can simply pass a pointer to a c_char_p object without specifying a length as restype argument, if the function output is a null terminated character array. Then you loop through the result until the null element is found, which indicates the end of the array.

So the following works beatifully for my use case:

# Define the function wrapper, the restype can simply be a
# pointer to c_char_p (without length!).
f = ctypes.CDLL('libgdal.so.20').GDALGetMetadata
f.restype = ctypes.POINTER(ctypes.c_char_p)
f.argtypes = [ctypes.c_void_p, ctypes.c_char_p]

# Prepare python result array.
result = []

# Call C function.
output = f(ptr, None)

# Ensure that output is not a null pointer.
if output:
    # Get first item from array.
    counter = 0
    item = output[counter]
    # Get more items, until the array accessor returns null.
    # The function output (at least in my use case) is a null
    # terminated char array.
    while item:
        result.append(item)
        counter += 1
        item = output[counter]