Auto Conversion of Structs to Dicts in Cython

Question

So, if you've got a header file.

%%file test.h

struct mystruct{
  int i;
  int j;  
};

And then you wrap it in Cython:

cdef extern from "test.h" nogil:
  struct mystruct:
    int i
    int j

And some function that returns back out to Py:

def spit_out_dict():
  return mystruct(5,10)

Cython correctly automatically generates a dict wrapper. However, when I wrap the original C header in a namespace, I haven't been able to get get Cython to still generate the dict wrapper correctly, something along these lines:

%%file test2.h

namespace outerspace{
struct mystruct{
  int i;
  int j;  
};
}

And Cython/Python:

cdef extern from "test2.h" namespace "outerspace" nogil:
  struct mynewstruct:
    int i
    int j

def spit_out_dict():
  return mynewstruct(5,10)

This won't compile -- lots of namespace complaint errors -- anyone experienced this before?

Answer 1

Your problem is that Cython seems to only expect namespaces to be used with cppclass. For structs, it generates some functions but just copies the full namespaced name in, causing errors:

static PyObject* __pyx_convert__to_py_outerspace::mystruct(struct outerspace::mystruct s);
                  ^
py_bit.cpp: In function ‘PyObject* __pyx_pf_6py_bit_spit_out_dict(PyObject*)’:
py_bit.cpp:721:15: error: ‘__pyx_convert__to_py_outerspace’ has not been declared

where it's trying to create a function called __pyx_convert__to_py_<classname>. (I think this might be worth submitting a bug report for.)

The trick in such circumstances is usually to lie to Cython. I create three files:

// test2.hpp
namespace outerspace{
struct mystruct{
  int i;
  int j;  
};
}

,

// test2_cy.hpp - a wrapper file purely for Cython's benefit
#include "test2.hpp"

using outerpsace::mystruct;

and the cython file

cdef extern from "test2_cy.hpp": # (I didn't test with "nogil", but it's probably fine...)
  struct mynewstruct:
    int i
    int j

def spit_out_dict():
  # for some reason using "return mystruct(5,10)" doesn't work, but this does...
  cdef mystruct a = mystruct(5,10)
  return a

Answer 2

This is a bug in Cython, fixed at https://github.com/cython/cython/commit/fa946e8435a4dcc3497fc7b0f4e87256d40844ba