passing python strings, through cython, to C

Question

I am trying to write a module with some c and some python parts. I am using cython to bridge the gap.

I want to store my (very long) string constants in python, because of the much nicer syntax:

const char long_string = "\npart of string\n"
  "next part\n"
  "last part\n";

versus:

long_string = """
part of string
next part
last part
"""

(the strings are much longer than this, and more complicated - to the extent that I don't want to have to add and remove the "s and \n"s every time I want to edit it with syntax highlighting. In fact, they are openCL kernels.)

I need to be able to turn these into c strings using cython, and according to the documentation I should just need this:

cdef bytes py_bytes = py_string.encode()
cdef char* c_string = py_bytes

and with no manual memory management, c_string will work as long as I keep a reference to py_bytes.

However, I can't get this working with a simple printf test. Here is my cython file:

cdef extern from "stdio.h":
  printf(char* string)

def go():
  py_string = """
a complicated string
with a few
newlines.
"""

  cdef bytes py_bytes = py_string.encode()

  cdef char* c_string = py_bytes

  printf(c_string)

  print "we don't get this far :("

which, when compiled at runtime using pyximport gives the following output to terminal before segfaulting:

a complicated string
with a few
newlines.
Segmentation fault: 11

now, I've checked what cython actually puts in the c file, and tried it in a vanilla C file where it doesn't segfault:

#include "stdio.h"

static char __pyx_k_1[] = "\na complicated string\nwith a few\nnewlines.\n";

int main(void) {
  void* output = printf(__pyx_k_1);
  if (!output) {
    printf("apparently, !output.");
  }
}

to be clear, cython generates code which catches the output of printf and tests for "not that". the type of the variable is a PyObject*.

My only guess here was that the string was improperly terminated, so printf just carries on past the end of it and causes the segfault, but since that doesn't happen in my pure c test, I'm completely stumped.

So, my actual question is how do I really pass a c-string to c code from cython? Answers pointing out an easier way to solve the actual problem I'm trying to solve at the top are also very welcome :)

Answer 1

Importing printf from libc.stdio fixes the problem for me:

from libc.stdio cimport printf

def go():
    py_string = """
a complicated string
with a few
newlines.
"""

    cdef bytes py_bytes = py_string.encode()
    cdef char* c_string = py_bytes
    printf(c_string)

    print "we actually got this far! :)"

The error is in the declaration of printf. That should have been, as stdio.pxd lists,

cdef extern from *:
    ctypedef char const_char "const char"

int printf(const_char *, ...)

whereas your version is implicitly object printf(char *); the default return value type is Python object rather than int as in C. Getting the right declaration turns off Cython's attempt to Py_XDECREF the return value from printf.

(Btw, in your "vanilla" C problem, you shouldn't be casting the return value from printf to void *.)